Vanguard Against Confusion

Feb 02 2011

Crowd-Sourced Relevancy Ranking in Bing or why Google is wrong

Let me start by saying I’m a Google fanboy and a half. I love that company and use all their products. My life revolves around my Nexus One and it’s deep and tight integration with Google’s stack which is my extremely large clustered brain-annex that’s constantly available with a few taps of my touchscreen.

Google is awesome and great and I love them to death. That said, they are wrong about Bing copying their results. 

See these first-party posts for details:

Thoughts On Search Quality (Bing Blog)

Bing uses Google’s Results — and denies it (Google Blog)

Why Microsoft is not in the wrong: Crowd Sourcing is not stealing

What Microsoft has done is create a genuinely useful process for improving the relevancy of the Bing search engine. It’s not a groundbreaking technique. Like most web based application in the world, they just monitor your behaviour. You type ‘foo’ into their search box, and are presented with 10 results on your first screen. You click #2 result. This is recorded. The same thing happens for 10,000 other users. Eventually Bing figures out that the #2 result really should be the #1 result, and ups it’s rank.

This is no different than Amazon’s suggestion engine ‘Users who viewed this product ultimately bought this other product’. 

This process doesn’t depend on Google. This process would work purely as a way of improving rank within the Bing system, with no outside influences. It also doesn’t really require a special toolbar to make it happen. You could collect that data through their normal web interface just as easily. 

This is not an issue of spyware, cheating, or copying. It’s just Bing using crowd-sourced data to determine relevancy. It’s very smart. It’s not dishonest. Get over it Google.

How to improve search result relevancy?

I find this interesting because, currently Google faces a huge challenge: Reducing the amount of spam that is polluting their results. Bing clearly has a tactic for that, though it may or may not be completely effective. The value of the click through data is only as good as the user who clicked on the result. If the majority of people searching for “Foo” clicked on say, option #3 instead of #2 in our previous example, and if #3 was a spam site, then #3 would get ranked up. Bad news!

Looks like Bing is trusting our judgement as a user community. We know what’s relevant and show it by clicking through. If we choose a spam site as our main result, Cest La Vie!!

But how can we improve search relevancy, and reduce false positives in our result set? The answer so far seems to be “curate the web”, or like Bing, use a “mechanical turk”, aka click stream crowd-sourced relevancy, assuming that people will be able to express their preference as a whole and emerge the correct answer over time.

Our solution: Contextual Search

My company has a different solution: Working at a higher level of abstraction than words and documents. We have built a novel search tool that allows users to search for contexts, not documents, and make decisions on contexts. 

Find me “license” where the document has “drivers license” in it, but not where it has “fishing license”, unless it also has something I want in it like “drivers license”. Traditional boolean search, which operates against an inverted index of terms to documents (which is what both Bing and Google offer), does not provide for this kind of decision making. It’s impossible without changing how the data is indexed, and that’s not anything these guys are going to be doing anytime soon. They have too much invested in their current methodology to change. 

We’re hoping to launch a public search site sometime this year that presents our novel approach to improving relevancy in search. I look forward to seeing how it performs compared to Google and Bing. 

More on that later, when it’s closer to reality. 

What do you think, world at large?

Does anyone have any other ideas about search relevancy? What are some other tactics one might employ, beyond Curating or Crowdsourcing? How else to make the spam go away?

#bing  #google  #search  #relevancy  #crowd sourced  #ranking 
Dec 29 2010

migration

moving here from there.

I decided to migrate the blog off of Blogger.com and onto Tumblr, for no particular reason other than Tumblr is simply cooler and has more features.

Some of tools I used were:

Import from Blogger to Tumblr Tool 

http://terrymhung.com/jtran/tumblr/import-blogger-to-tumblr.php

Syntax Highlighter from Alex Gorbatchev

http://alexgorbatchev.com/SyntaxHighlighter/

Disqus.com for comments support

http://www.howtogeek.com/howto/22032/add-traditional-comments-to-tumblr-blogs-with-disqus/

#blogger.com  #tumblr.com  #migration 
Aug 02 2010

010 Editor: My new favourite tool

During the course of my work, I use a hex editor a lot.

Specifically, I use a hex editor most for reverse engineering binary file formats that have no documentation or for fixing corrupted files, and the like. One thing that I’ve always wanted, was some way to view the binary contents as structured data.. Like “Starting at this byte offset, consider the next four bytes to be an integer, and show me that integer, then, using that integer, take that many bytes immediately following it, consider them to be string data, and decode as UTF8 or EBCDIC… etc.

All that is fairly complex, and generally well beyond the facilities of anything short of a fairly low level and full-featured programming language.

Well, looks like the folks over at SweetScape realized this is a workflow that at least SOME people need to have… and so they built the perfect tool for doing that. It’s called the 010 Editor. It has this great feature called Binary Templates as well as scripts. It’s the bomb, and it’s accelerated my reverse engineering work by at least an order of magnitude.

I’m buying a license, and anyone know know me will realize that’s a pretty big deal. I’m a big fan of FOSS and generally try to use as much of it as possible, avoiding commercial apps… but this is a big exception. SweetScape is a small company run by a father and son team. Hardly “The Man”. Lowell and Graeme Sweet — you rock the block and treat the bytes right. ;)

  /   Source: vanguard-against-confusion.blogspot.com

#010editor  #a better way  #binary templates  #hex editor  #reverse engineering  #SweetScape  #tools 
Feb 12 2010

Big Data

When I was a kid — 7 years old — I got my first computer. At that time, using a computer meant programming a computer. My Commodore VIC20 didn’t have a storage device when I got it (I got a tape drive later on). So, if you turned it off, the in memory program was gone. You had to type it in again the next time you started up the computer.

I remember learning about programming. One of my first programs was just:

10 PRINT “TROY”

I was delighted to see my name show up on the screen. The computer knew me now. I was being reflected back from the screen in glowing pixels. I’d taken a little bit of me, and put it into the computer, and the computer was letting me look at it, and learn about myself, and learn about the computer, at the same time. I was hooked.

10 FOR I = 1 TO 10
20 PRINT “TROY”
30 NEXT

This blew my mind. While there was something magical about seeing my name on the screen the first time. Seeing it ten times was just really really exciting. Something about the quantity was just really cool… and how FAST it happened. It put my name up there ten times as fast as it put it up there one time!! Thrilling!!

10 PRINT “TROY”
20 GOTO 10

The screen was filled with my name…. and it didn’t stop. I got up and I ran to tell my mom. I drug her downstairs into the basement and babbled on about the computer and how it was just going and going, and how I did it. GOTO mom… GOTO! My excitement was more than I could contain. I was officially a rocket scientist now. My excitement, like the program was INFINITE!!! I created something that was endless, infinitely long. There was no way to count how many times it put my name on the screen. My computer, at that moment, knew only one thing — my name, my program, and it would run it forever until I told it to stop. Happily. This was a kind of love and dedication that was far beyond what any person could ever give. There was something really deep here, between me and the computer.

Fast forward now, 24 years later. I’m a software engineer for a living now, and have been for a while. At my company, I recently got promoted to a fancier title “Director of Software Development”, and all the responsibility for success lies on my shoulders. People answer my job advertisements with the salutation “Mr. Howard”. That part freaks me out.

I’m still excited by big data. That infinite loop of TROYs on my screen was just the start. Now I design systems that process terabytes of data at a time on hundreds of servers. One of the most fascinating parts of my job these days is still the same as when I was a kid. I love hitting “Run” on a unit test, and seeing what happens. I feel good when it’s successful once. My next step, almost without fail, is to see what happens when it runs ten times in a row.. Then 100… Then 1000… 10000… 100000… I just keep adding zeros until the thing breaks down, or until I get bored with it.

Big Data is still exciting, still fascinating. I’ve now given my computer programs more interesting sample material to work from, and so their world view has expanded. Now instead of only knowing me and my name, my programs know all the details of the personal and business lives of thousands of people whose email is processed by the programs. I think my computer still loves me more than any of them though. Secretly, somewhere in there, I know there’s an infinite loop on a background thread that’s just cycling over the string “TROY”… forever.

  /   Source: vanguard-against-confusion.blogspot.com

#big data 
Oct 23 2009

An Infinite Stream Of Bytes

No, I’m not about to wax poetic about the deep ontological issues raised in The Matrix, or speak meaningfully about how transient the modern world of communication is and how the artifacts of our lifetime have become ephemeral such that our posterity will not be able to remember us, even if they wanted to.

Instead I’m going to post a code snippet that solves an annoying little scenario that comes up every now and again when writing parsers.

Basically, it goes like this:

You’re writing a parser, and you need to check every byte in a stream of bytes coming from a file/network/etc.. You might need to read forward or read backward a little, to match a multi-byte pattern or value within n bytes of another value. You figure instead of “peeking and seeking” against the stream (what it’s read-only!?!?), your parser can just stored the state, and still only look at a single byte at a time. That’s great and all, and you do a quick implementation using stream.ReadByte, which seems to work…

Except it’s slow. You know from experience that block reads are way faster, and you want to read a block of data that’s say 1k or 4k from your stream, and then parse that, fetch another block, parse that, etc… but what if your pattern straddles two blocks? What if the first byte of a two byte sequence is the last byte in a block and the next block’s first byte is the second character? Now your parser needs to stop what it’s doing, exit the loop, go grab some more data, then restart it’s iteration over that.. You could build all that behaviour into your parser (for every parser that you write).. but it’s non-trival to deal with. In fact it’s a real pain in the butt to refactor a parser to work that way.

Also, you think to yourself “Man… It would be SOOOOooooo much nicer if I could just write a foreach loop, and like get every byte in the stream in one bit long iteration… Why doesn’t System.IO.Stream implement IEnumerable?!?” It totally makes sense that it should…

Anyhow, story’s over. Here’s the code to solve it:


public static IEnumerable<byte> GetBytesFromStream(Stream stream)
{
const int blockSize = 1024;

byte[] buffer = new byte[blockSize];
int bytesRead;

while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
for (int i = 0; i < bytesRead; i++)
{
yield return buffer[i];
}
}
}



And in case it’s not obvious, I’ll explain what this little guy does. It does a block read from the stream (adjust your blocksize to suit or make it a parameter), iterates over the block, uses the yield keyword to return bytes via the IEnumerable<T> interface. The while loop checks the return value of stream.Read() to see if it returns zero, which means, basically, the stream is done (EOF). If there was a partial read (e.g. less than your blocksize buffer) bytesRead will be the amount that DID successfully read, and so your for loop that is iterating over the block uses bytesRead to ensure we only return valid data (if we had used buffer.Length or blockSize, and had a partial read, the stuff after the “new data” would be data from the last read. NOT COOL!).

You could stick this method in your utility class if you’d like, or make a wrapper class that wraps Stream and implements IEnumerable<byte>… whatever you want. Maybe you want to be all modern and cool and make it an extension method for Stream.

Here’s an example wrapper class:


public class EnumerableStream : Stream, IEnumerable<byte>
{
private readonly Stream _baseStream;

public EnumerableStream(Stream stream)
{
_baseStream = stream;
}

public IEnumerator<byte> GetEnumerator()
{
var bytes = GetBytesFromStream(_baseStream);
return bytes.GetEnumerator();
}

IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}

private static IEnumerable<byte> GetBytesFromStream(Stream stream)
{
const int blockSize = 1024;

byte[] buffer = new byte[blockSize];
int bytesRead;

while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
for (int i = 0; i < bytesRead; i++)
{
yield return buffer[i];
}
}
}

public override bool CanRead
{
get { return _baseStream.CanRead; }
}

public override bool CanSeek
{
get { return _baseStream.CanSeek; }
}

public override bool CanWrite
{
get { return _baseStream.CanWrite; }
}

public override void Flush()
{
_baseStream.Flush();
}

public override long Length
{
get { return _baseStream.Length; }
}

public override long Position
{
get
{
return _baseStream.Position;
}
set
{
_baseStream.Position = value;
}
}

public override int Read(byte[] buffer, int offset, int count)
{
return _baseStream.Read(buffer, offset, count);
}

public override long Seek(long offset, SeekOrigin origin)
{
return _baseStream.Seek(offset, origin);
}

public override void SetLength(long value)
{
_baseStream.SetLength(value);
}

public override void Write(byte[] buffer, int offset, int count)
{
_baseStream.Write(buffer, offset, count);
}
}



And an example of the extension method way…


public static class StreamExtensions
{
public static IEnumerable<byte> GetBytes(this Stream stream)
{
const int blockSize = 1024;

byte[] buffer = new byte[blockSize];
int bytesRead;

while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
for (int i = 0; i < bytesRead; i++)
{
yield return buffer[i];
}
}
}
}



Enjoy!

  /   Source: vanguard-against-confusion.blogspot.com

#Stream  #refactoring  #string  #byte  #c  #a better way  #code snippet  #.NET  #IEnumerable 
Jul 23 2009

Unicode string detection

I had the need to detect wether or not a given string (in .Net/C#) was unicode or not.. Specifically filenames. I had a situation where a filename might be passed to me, that could possibly contian unicode. If it DID contained unicode characters, I needed to run GetShortPathName and get the 8.3 filename for the file, before passing it into a legacy component that couldn’t handle unicode names…

Well, a “big hammer approach” might just call GetShortPathName on every filename, just to be sure… But that’s a costly API call if your having to do this a million times a second.

So, long story short, I wrote this little function to detect unicode in a c# .Net string:


public static bool IsUnicode(string s)
{
return s != Marshal.PtrToStringAnsi(Marshal.StringToHGlobalAnsi(s));
}



Now homework for all you kiddies out there… Is this code a memory leak? If so, what should you do to fix it? If not, why not?

  /   Source: vanguard-against-confusion.blogspot.com

#globalization  #hGlobal  #leak  #memory  #string  #c  #detection  #hackish  #unicode  #code snippet  #Marshal  #.NET 
Jan 06 2009

Switch To…, Retry, Cancel

Well, as much as I hate to admit it, I was recently working on VB6 application that uses Office COM automation. It needed to have Word and Excel do a few things while the main application waited.

Every time a user clicked on the UI, you’d get that beautiful dialog “This application cannot be completed because the other application is busy. Choose ‘Switch To’ to activate the busy application and correct the problem.

This is particularly onerous because if the user does not click switch to, basically, both programs, the VB6 app AND the Office app sit around waiting for the user to do something. Yay!

Luckily I found the quick and dirty answer to this. Somewhere… Anywhere before the point that you make the COM call, just insert these two lines:


App.OleRequestPendingTimeout = 2147483647
App.OleServerBusyTimeout = 2147483647



… and you never have to see that dialog again.

Enjoy!

  /   Source: vanguard-against-confusion.blogspot.com

#VB6  #Ole Automation Procedures  #COM 
May 15 2008

How to get information about your current culture.

Instead of doing a college survery and asking a bunch of probing questions about the lives of twenty-somethings, there’s an easier way to get information about your current culture. Just look at CultureInfo.CurrentCulture.

Here’s a quick program that explains how to do that. This can be very useful in debugging and troubleshooting how your program behaves on machines that are setup for other laungages or regions.


using System;
using System.Collections.Generic;
using System.Text;
using System.Globalization;

namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
CultureInfo currentCulture = CultureInfo.CurrentCulture;

Console.WriteLine("CultureInfo");
Console.WriteLine("-----------");
Console.WriteLine("DisplayName: {0}", currentCulture.DisplayName);
Console.WriteLine("Name: {0}", currentCulture.Name);
Console.WriteLine("LCID: {0}", currentCulture.LCID);
Console.WriteLine();

Console.WriteLine("NumberFormatInfo");
Console.WriteLine("----------------");
Console.WriteLine("Decimal Seperator: {0}", currentCulture.NumberFormat.NumberDecimalSeparator);
Console.Write("Digits: ");

foreach (string s in currentCulture.NumberFormat.NativeDigits)
{
Console.Write(s + " ");
}

Console.WriteLine();
}
}
}




Base output should look like:


CultureInfo
-----------
DisplayName: English (United States)
Name: en-US
LCID: 1033



NumberFormatInfo
----------------
Decimal Seperator: .
Digits: 0 1 2 3 4 5 6 7 8 9

  /   Source: vanguard-against-confusion.blogspot.com

#globalization  #c  #CultureInfo  #CurrentCulture  #.NET 
+

Refactoring a big if block into a simple command processor using attributes

Recently someone had a problem where they had some massive control block full of if statements looking at a string, dispatching one of a variety of functions. The if block was massive. Hundreds of if statments, hundreds of magic strings.

Interestingly all the functions had the same signature… So I gave him this example of how to use attributes on the methods to specify the corresponding token, then we use Reflection to scan the assembly for all the functions with that attribute, then create a function table keyed by thier token, to privde fast lookup. This example shows how to creat an object instance and then invoke the method via reflection, but this could be made much simpler if the methods were all static and the function protoype was part of an interface instead of just a unspoken convention.

Here’s the “Before” example from the original question…


string tag;
string cmdLine;
State state;
string outData;

...

if (token == "ABCSearch") {
ABC abc = new ABC();
abc.SearchFor(tag, state, cmdLine, ref outData);
}
else if (token == "JklmDoSomething") {
JKLM jklm = new JKLM();
jklm.Dowork1(tag, state, cmdLine, ref outData);
}



A couple of notes:


  • There is no correlation between the token and the class name (ABC, JKLM, …) or the method (SearchFor, Dowork1).
  • The methods do have the same signature:
    void func(string tag, State state, string cmdLine, ref string outData)
  • The if ()… block is 500+ lines and growing




And here is my example command processor (as a console app):


using System;
using System.Collections.Generic;
using System.Reflection;

namespace ConsoleApplication2
{
public class Program
{
static void Main(string[] args)
{
while(true)
{
Console.Write("[e(x)ecute, (t)okens, (q)uit] -> ");
string s = Console.ReadKey().KeyChar.ToString().ToLower();
Console.WriteLine();

switch (s)
{
case "q":
Console.WriteLine("Finished.");
return;

case "t":
Console.WriteLine("Known tokens:");
foreach (string tokenName in CommandProcessor.GetTokens())
{
Console.WriteLine(tokenName);
}
break;

case "x":
string token = string.Empty;
string tag = string.Empty;
string cmdLine = string.Empty;
string state = string.Empty;

Console.Write("token: ");
token = Console.ReadLine();
Console.Write("tag: ");
tag = Console.ReadLine();
Console.Write("cmdLine: ");
cmdLine = Console.ReadLine();
Console.Write("state: ");
state = Console.ReadLine();

try
{
string output = CommandProcessor.DoCommand(token, tag, cmdLine, State.GetStateFromString(state));
Console.WriteLine("Output:");
Console.WriteLine(output);
}
catch (TokenNotFoundException ex)
{
Console.WriteLine(ex.Message);
}
catch (Exception ex)
{
Console.WriteLine("Unknown error occured during execution. Exception was: " + ex.Message);
}
break;

default:
Console.WriteLine("Unknown command: {0}", s);
break;
}
}
}
}

public class CommandProcessor
{
// our dictionary of method calls.
internal static Dictionary availableFunctions = new Dictionary();

static CommandProcessor()
{
SetupMethodCallDictionary();
}

private static void SetupMethodCallDictionary()
{
// get the current assembly.
Assembly assembly = Assembly.GetExecutingAssembly();

// cycle through the types in the assembly
foreach (Type type in assembly.GetTypes())
{
// cycle through the methods on each type
foreach (MethodInfo method in type.GetMethods())
{
// look for Token attributes on the methods.
object[] tokens = method.GetCustomAttributes(typeof(TokenAttribute), true);

if (tokens.Length > 0)
{
// cycle through the token attributes (allowing multiple attributes
// leaves room for backwards compatibility if you change your tokens
// or consolidate functionality of the methods. etc.
foreach (TokenAttribute token in tokens)
{
// look for the token in the dictionary, if it's not there add it..
MethodInfo foundMethod = default(MethodInfo);
if (availableFunctions.TryGetValue(token.TokenName, out foundMethod))
{
// if there is more than one function registered for the same
// token, just keep the last one found.
availableFunctions[token.TokenName] = method;
}
else
{
// add to the table.
availableFunctions.Add(token.TokenName, method);
}
}
}
}
}
}

public static string DoCommand(string token, string tag, string cmdLine, State state)
{
// the data returned from the command
string outData = string.Empty;
MethodInfo method = default(MethodInfo);

// see if we have a method for that token
if (availableFunctions.TryGetValue(token, out method))
{
// if so, create an instance of the object, and then execute the method,
// unless it's static.. in which case just execute the method.
object instance = null;
if (!method.IsStatic)
{
// this just invokes the default constructor... if you need to pass
// parameters use one of the other overloads.
instance = Activator.CreateInstance(method.ReflectedType);
}

object[] args = new object[] { tag, state, cmdLine, outData };

method.Invoke(instance, args);
outData = (string)args[3];
}
else
{
throw new TokenNotFoundException(string.Format("Token {0} not found. Cannot execute.", token));
}
return outData;
}

public static IEnumerable GetTokens()
{
foreach (KeyValuePair entry in availableFunctions)
{
yield return entry.Key;
}
}
}

public class State
{
public State(string text)
{
_text = text;
}

private string _text;

public string Text
{
get { return _text; }
set { _text = value; }
}

public static State GetStateFromString(string state)
{
// implement parsing of string to build State object here.
return new State(state);
}
}

[AttributeUsage(AttributeTargets.Method)]
public class TokenAttribute : Attribute
{
public TokenAttribute(string tokenName)
{
_tokenName = tokenName;
}

private string _tokenName;

public string TokenName
{
get { return _tokenName; }
set { _tokenName = value; }
}
}

[global::System.Serializable]
public class TokenNotFoundException : Exception
{
//
// For guidelines regarding the creation of new exception types, see
// http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpgenref/html/cpconerrorraisinghandlingguidelines.asp
// and
// http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dncscol/html/csharp07192001.asp
//
public TokenNotFoundException() { }
public TokenNotFoundException(string message) : base(message) { }
public TokenNotFoundException(string message, Exception inner) : base(message, inner) { }
protected TokenNotFoundException(
System.Runtime.Serialization.SerializationInfo info,
System.Runtime.Serialization.StreamingContext context)
: base(info, context) { }
}

public class ABC
{
[Token("ABCSearch")]
public void SearchFor(string tag, State state, string cmdLine, ref string outData)
{
// do some stuff.
outData =
string.Format("You called ABC.SearchFor. Parameters were [tag: {0}, state: {1}, cmdLine: {2}]", tag, state.Text, cmdLine);

}
}

public class JKLM
{
[Token("JklmDoSomething")]
public void Dowork1(string tag, State state, string cmdLine, ref string outData)
{
// do some other stuff.
outData =
string.Format("You called JKLM.Dowork1. Parameters were [tag: {0}, state: {1}, cmdLine: {2}]", tag, state.Text, cmdLine);
}
}
}

  /   Source: vanguard-against-confusion.blogspot.com

#attributes  #refactoring  #c  #command processor  #.NET  #reflection 
+

Filtering a network stream using a wrapper

So not that long ago, someone posted a question asking how to deal with a certain situation. The situation is such that there is a network file stream coming from somewhere, that has certain data you want to keep, and certain data you don’t want to keep. Control blocks, extra header information, weirdo protocol, too much data coming back form an API, etc..

My suggestion was to create a simple container object (aka wrapper) to the existing network stream, that operates the same as the network stream, but does the necessary filtering.

Here’s an example of how you’d use it, and and example base class implementation of for the filters follows it. In the actual problem case example, he was dealing with a NetworkStream that contained Xml data in irregular chunks, with control blocks as fixed headers. Each header indicates how much XmlData follows. The filter will remove the headers as needed, presenting a simple stream of Xml data to the XmlReader to parse.

I’ve left out the concrete implementation that actually parses the stream, and here you just have the FilteredNetworkStream base class and an idea of how to use it once you implement it. All that’s left for the implementer is to override the abstract method FilterBeforeRead, which contains the customized filtering logic for the particular situation.



using (NetworkStream inputStream = GetNetworkStreamFromSomewhere())
using (StreamWriter outputStream = new StreamWriter(@"C:\Path\To\File.xml", false))
{

XmlReader reader = XmlReader.Create(new FilteredNetworkStream(inputStream));
while (reader.Read())
{

// method returns empty string if current data is discardable
string outputData = GetDesiredDataFromReader(reader);

if (!string.IsNullOrEmpty(outputData))
{

// save desired data to local file
outputStream.Write(outputData);
}
}
}



Here’s the base class:


public abstract class FilteredNetworkStream : Stream
{
public FilteredNetworkStream(NetworkStream baseStream)
{
_baseStream = baseStream;
}

protected NetworkStream _baseStream;
public abstract void FilterBeforeRead();

#region Stream Implementation

public override bool CanRead
{
get { return _baseStream.CanRead; }
}

public override bool CanSeek
{
get { return _baseStream.CanSeek; }
}

public override bool CanWrite
{
get { return _baseStream.CanWrite; }
}

public override void Flush()
{
_baseStream.Flush();
}

public override long Length
{
get { return _baseStream.Length; }
}

public override long Position
{
get
{
return _baseStream.Position;
}
set
{
_baseStream.Position = value;
}
}

public override int Read(byte[] buffer, int offset, int count)
{
this.FilterBeforeRead();
return _baseStream.Read(buffer, offset, count);
}

public override long Seek(long offset, SeekOrigin origin)
{
return _baseStream.Seek(offset, origin);
}

public override void SetLength(long value)
{
_baseStream.SetLength(value);
}

public override void Write(byte[] buffer, int offset, int count)
{
_baseStream.Write(buffer, offset, count);
}

#endregion
}

  /   Source: vanguard-against-confusion.blogspot.com

#Stream  #abstract  #c  #container  #NetworkStream  #pattern  #.NET  #Filter  #wrapper 
Apr 29 2008

Getting the field list returned from an ad-hoc Sql query

So, recently I needed to make an application that allowed a user to enter an arbitrary SQL query, and elsewhere in the UI I needed to display a drop-down with the fields that this arbitrary query returned.

This poses a small problem. It’s very simple if the user is doing simple queries, that don’t take long to execute. You could just run the query, then, take the first result, and get the list of fields. Well.. This works for simple queries that return small result sets, but we needed to put in queries that potentially return as many as 48 million results, using complex queries including joins between multi-million rowed tables, aggregates, and that sort of thing..

In other words, the queries are slow. Really slow. They create a lot of UI lag when I go to get the field names for the drop down box.

My first attempt was to take the query and wrap it up like this:

SELECT TOP(1) * FROM ( // original query here // ) fieldNamesTable

My thinking was that if I specified that I only wanted the first record it’s be really quick, even with a complex query. This is true. It’s must faster, but it’s still slow. Too slow. A lot of UI lag still remained.

So, my second attempt worked much better. I wrapped the query again, but now it looks like:

SELECT * FROM ( // original query here // ) fieldNamesTable WHERE 1 = 0

Instead of specifying I wanted the first record, I put a phrase in the WHERE clause that will always be false. The SQL Server’s query execution engine realizes that, and so it knows that the query will never be able to return data. So it immediately returns with 0 results. But I get the field names!! This is SUPER fast!

Enjoy,
Troy

  /   Source: vanguard-against-confusion.blogspot.com

#Microsoft Sql Server  #MSSQL  #SQL 
Mar 24 2008

An amusing bought with the DataGridView control

As usual, WinForms GUI programming is a terrible PIA. Even worse is the flagship of all controls the great beast known as the DataGridView. Working with the DataGridView bends your mind like offensive cutlery at Uri Geller’s dinner table.

During my most recent encounter with this control of the third kind, I needed to use a ComboBox column, and that column needed to have an effect on the contents of the other controls. Normally, you could just hook up a cellvaluechanged event or something of that nature, but that doesn’t work out on a ComboBox control in a DataGridView column… There’s no event that would fire when a user selected an item in the dropdown. Only after the user selected the item, then refocused on some other control.

That was annoying. Too many clicks for the user! When I select the item in the dropdown, the row should react, I shouldn’t have to click elsewhere.

So here’s a quick example of what I did to get that working. In the example, we handle the datagridview’s EditControlShowing event, then grab a reference to the combobox, then unwire any previous events we may have hooked up to SelectionChangeCommitted, then wire the event. In the SelectionChangeCommitted event we call _dataGridView.EndEdit() to effect the other rows.

Enjoy!


public class Example
{
///
/// Constructor
///
public Example()
{
_dataGridView = new DataGridView();

// setup the datagridview here.
DataGridViewComboBoxColumn fooColumn = new DataGridViewComboBoxColumn();
fooColumn.Name = "Foo";
fooColumn.ValueType = typeof(String);
fooColumn.HeaderText = "Foo";
fooColumn.Items.Add("Bar");
fooColumn.Items.Add("Baz");
fooColumn.Items.Add("Fizz");
fooColumn.Items.Add("Buzz");
fooColumn.Items.Add("FizzBuzz");
fooColumn.DefaultCellStyle.NullValue = "Bar";

_dataGridView.Columns.Add(fooColumn);

// hook up editing control showing event
_dataGridView.EditingControlShowing += new DataGridViewEditingControlShowingEventHandler(_dataGridView_EditingControlShowing);

// create a delegate for the method that will handle the event
_comboBoxSelectDelegate = new EventHandler(combo_SelectionChangeCommitted);
}

private DataGridView _dataGridView;
private EventHandler _comboBoxSelectDelegate;

void _dataGridView_EditingControlShowing(object sender, DataGridViewEditingControlShowingEventArgs e)
{
// get the control from the event args.

ComboBox combo = e.Control as ComboBox;


if (combo != null)
{
// remove the event subscription if it exists.
combo.SelectionChangeCommitted -= comboSelectDelegate;

// add a subscription to the event
combo.SelectionChangeCommitted += comboSelectDelegate;
}
}

void combo_SelectionChangeCommitted(object sender, EventArgs e)
{
// handle the event, and end edit mode
_dataGridView.EndEdit();
}
}

  /   Source: vanguard-against-confusion.blogspot.com

#combo  #SelectionChangeCommitted  #events  #c  #DataGridView  #box  #DataGridViewComboBoxColumn  #combobox  #.NET  #EditingControlShowing 
Mar 08 2008

Black Box OMG it’s addictive.

BlackBox - A simple puzzle game where you shoot rays of light into a black box, and determine the location of atoms inside the box based on the entry and exit points of the ray.

Seriously addicting.

Here is a Wiki article about it and an online-playable Flash version.

  /   Source: vanguard-against-confusion.blogspot.com

#game  #logic  #blackbox  #black  #addictive  #box  #puzzle 
Feb 13 2008

Sharing Menu Items between ToolStrips on a Windows Form

So, you may have found yourself building a nice user-friendly, somewhat complicated Windows Forms application, that had lots of drop-down menus and right click context menus, and what not.. You may have naively assumed that you could *share* your menu items, so that you have a consistent set of options, icons, and more importantly event handlers for a particular menu item or set of menu items.


Well, I did… I had a Tools drop-down menu with some basic functions that I wanted to also be accessible from a right-click content menu on a treeview. Redundant? Sure… Convenient? Definately.

There’s a little annoying detail that says that a ToolStripMenuItem can’t be “owned” by more than one ToolStrip. In other words, it can only be in one place at a time. So, when you do something like this:

toolsToolStripMenuItem.DropDownItems.Add(myMenuItem);
treeNodeContextToolStrip.Items.Add(myMenuItem);

The menu item in question suddenly disappears from the tools menu, and appears in the content menu… Hmm…

So, in order to share the menu item, I came up with this hackish solution….I handled the Opening event for the two menu strips and in each one, “took ownership” of the menuitem. So, through sleight-of-hand it seems to exist in one place at a time. We can only get away with this because, ToolStrips, being modal, only show one at a time.

So, here’s a simple sample:

Hot-Swap Menu Item Sample


private void treeNodeContextMenuStrip_Opening(object sender, CancelEventArgs e)
{
treeNodeContextMenuStrip.Items.Insert(3, myToolStripMenuItem);
}

private void toolsToolStripMenuItem_DropDownOpening(object sender, EventArgs e)
{
this.toolsToolStripMenuItem.DropDownItems.Insert(5, this.myToolStripMenuItem);
}

  /   Source: vanguard-against-confusion.blogspot.com

#ContentMenuStrip  #hackish  #Windows  #Forms  #ToolStrip  #ToolStripMenuItem  #.NET 
Nov 15 2007

ASP.NET Gridview SelectIndexChanged event not firing

If, you find yourself in the unfortunate position of having a dynamically created Gridview Control in your ASP.NET page AND needing to handle the SelectedIndexChanged event… as I was, you may find yourself banging your head against the monitor trying to understand why the event isn’t getting fired.


Well, dear reader, that’s because you didn’t set the Gridview.ID property, and when the callback for the event is fired, it does that by ID/parameters.. The parameters are there, but the ID isn’t.. So it can’t find your control, so it can’t fire the event.

So, always assign the ID of dynamically generated ASP.NET controls!

  /   Source: vanguard-against-confusion.blogspot.com

#Gridview  #events  #SelectedIndexChanged  #ASP.NET  #Control 
Page 1 of 2