Programinator

Holy Smokes!

Thursday, March 25, 2010

Storing Passwords

I’ve never much considered myself a security expert, and I’ve rarely paid much attention to how much attention I’ve paid to things like storing passwords.

I’ve always had a ‘just protect the innocent’ when it comes to that stuff.

But, there comes a time when you have to step up to the plate, and learn the proper way of doing something.  And I’m starting to focus on, how to properly store passwords, for a scenario where an authentication system exists, which is exposed to the web.

You know; I guess I would normally look at it as, how important is the data that exists on the application?  Would someone want to take the time to obtain such passwords?

Then I realize, this isn't really the point.  Let’s not even consider the fact that people will often use the same password over and over, so the act of obtaining a password and an email address together could put an awful lot of risk.  Yes, I now believe that using a different password in different places is probably a good idea.

So, I looked into it, and I decided, why store the password?  Why not run that password though a Message Digest Algorithm, like MD5.  The thought was brilliant!  It works!  By not storing the password, I’ve essentially removed this burden.  See a stored hash does not contain any traces of the original password, and so this basically means that someone would have to have the hash, and start brute forcing the same algorithm until they have the same result.  right?

Wrong!

As I thought about all the hassle that someone would have to go through to generate possible passwords, it dawned on me, that by caching that which was generated, one would essentially be able to create a dictionary, whereby getting a hold of a hash, they would be able to come back very quickly with the original password, or, at the very least, a lucky collision, generating a sum that the application would accept.

I stopped here… to do some research.

Turns out, some nice folks on the internet (trillions and zillions of people) have already thought of this, and use an analogy called ‘salting’.  Forgive my improper use of terms, I’ve already stated I’m just learning. =)

So, what salting is, is taking something known, and appending it to the password before generating the hash.

See, in this case, even if a password is discovered that generates the hash in the database, entering that through the application will fail, simply because:

original password hash generated as:
     password_hash = hash(password + salt)

When discovered password is tried again
    password_hash = hash(password_and_salt + salt)

Well, that’s the theory anyways.  So I came up with a really redundant way of doing this, and I think the net effect would have been fine.

Although, I talked to a friend of mine, much more knowledgeable in the area. I immediately stated that I have no idea what I’m doing, but I have an interest to learn.  He points out my flaws.

First of all, I hard coded the salt, which is fine so long as it’s kept private, but once that salt is discovered, ALL passwords are compromised.

My reaction was, OK, so generate a random salt for each password, where do I store them?  They can’t be public!

Sure they can.  It takes a long time to generate a dictionary, and if the salt is large enough, and the passwords are moderately strong, the time and space requirements to generate these reverse lookup or rainbow (two very different things) tables makes it annoying, or very likely un feasible for a thief.

So I’m now sitting in a place, where I know a lot more about the topic.  I’m still not an expert, and I’m still VERY much willing to learn.  I’ll share my little test project with anyone willing to look, and very much welcome all criticism.  I figure it’s hard to troll someone who’s openly stating their desire to learn.  So, have at it!

I set a table up in SQL Server that looks like this

id -  int 
username - varchar(50)
password - binary(32)
salt - binary(64)





I set up 64 bytes for the salt.  Which is probably overkill.  Would very much be interested to hear how large you folks would use.  Maybe only 8 bytes?





I would use something like the following to generate salts and password hashes.



 /// <summary>
/// Password Encoding
/// </summary>
private static Encoding m_encoding;

/// <summary>
/// Random number generator
/// </summary>
private static Random m_random;

/// <summary>
/// Hash Provider
/// </summary>
private static SHA256 m_sha256;

/// <summary>
/// initialize static members
/// </summary>
static Security() {
m_encoding = Encoding.UTF8;
m_random = new Random();
m_sha256 = new SHA256Managed();
}

/// <summary>
/// Generate new random salt
/// </summary>
/// <returns></returns>
private byte[] new_salt() {
byte[] salt = new byte[64];
m_random.NextBytes(salt);
return salt;
}

/// <summary>
/// Generate Hash, in a more english form
/// </summary>
/// <param name="input">password to hash</param>
/// <param name="salt">must be binary, because no constraints to salt, doesnt
/// have to be characters, nor compatable with specified encoding</param>

/// <returns>literal hash</returns>
private string get_hash(string input, byte[] salt) {
StringBuilder sb = new StringBuilder();
foreach (byte b in get_hash(m_encoding.GetBytes(input), salt))
sb.Append(b.ToString("x2").ToLower());
return sb.ToString();
}

/// <summary>
/// Generate Hash
/// </summary>
/// <param name="input">decoded password</param>
/// <param name="salt">salt</param>
/// <returns>binary hash</returns>
private byte[] get_hash(byte[] input, byte[] salt) {
MemoryStream ms = new MemoryStream();
ms.Write(input, 0, input.Length);
ms.Write(salt, 0, salt.Length);
return m_sha256.ComputeHash(ms.ToArray());
}



Nothing too technically complicated.



When creating users, I would store the password like this:



byte[] salt = new_salt();
command.Parameters.AddWithValue("@username", username);
command.Parameters.AddWithValue("@password",
get_hash(m_encoding.GetBytes(password), salt));
command.Parameters.AddWithValue("@salt", salt);



So as you can see, we never actually store the password.  Once the password is entered, its hashed, and then forgotten.  Some might say that this is a weakness, especially for things like password recovery.  I say it’s fine, who wants a password they forgot anyways.  And if I were able to regenerate it, so could a thief.



Something to keep in mind here, to authenticate this, all we need is the salt.  And this is NOT a weakness, because generating the hash, is like a one way encryption.  I can take any piece of data, and generate a hash, but I cannot determine the data to get to a hash.



I would authenticate like this.



first obtain the salt.



SELECT salt FROM [users] WHERE [username] = @username



Next, use that salt to generate a new password hash, the same as we did when we created the user in the first case.



command.Parameters.AddWithValue("@username", username);
command.Parameters.AddWithValue("@password", get_hash(m_encoding.GetBytes(password), salt));


Then I would use this to hit the database again, with a scalar query.



SELECT ISNULL ( (SELECT 1 FROM [users] WHERE [username]=@username AND [password]=@password), 0)


The nice thing about doing it this way, is that I never pull anything related to the password into the application, just the salt, and just when authenticating. I would never store any of it in an object relational model of any sort. So.. someone grade me! Have I finally found a relatively sane way to store passwords? As a matter of fact, can someone tell me what the original password is here?



password hash:



0x50B0B41BF017A921730B950F8195AB9E7BA8FC525509B20851BFB12E1FB328E5



salt: 



0x724670D30F17831437162D383AC4DFC43BF9A02C9566D6126F7829636188A556
6614A4B0302FF3228A7A39403EDBE76FFF7086CC2701410B6F5D6497ED1FB3AF

Sunday, April 26, 2009

Never Underestimate the power of Google

Ok, I guess we’ve all been there before.  Need to know something?  Where should I look?

Where should I ask… where else?  Google of course!

This actually has nothing to do with programming, though maybe a programming subculture.

I have a Wii, wanted to play the Wii, and crap, i was denied the service.  Couldn’t understand it.  The burger just wouldn't turn on.  Not power LED, no nothing.

As a professional analyst, I naturally went through the best diagnostic checklist I could come up with after 5 or so beers.

  • plugged in?
  • outlet giving juice?
  • too drunk to turn it on?

Nope, it’s messed.

So.. naturally, I considered my resolution options.

  • Tech Support.
  • Warranty
  • Attempt to return it at Walmart (even though it was bought elsewhere)

None of these options left me with dignity.  I had only one course of action, my old friend Google.

Turns out, you must unplug power source from outlet AND unit for 90 seconds.  Such an act causes an internal breaker to reset, its a safety mechanism.

This leaves me with a few ideas on a new thesis.

  • DRM (Drunk operation Retention Management)
  • maybe I shouldn’t plug the unit into an outlet controlled by a light switch?
  • this is obviously the type of user interaction I need to implement in my own products

Sorry for the rant.  I’m actually happy I didn’t have to resort to any of the above.  I just thought this bit of info would either be enjoyable to you, or possibly helpful should you ever find yourself 5 beers in, and wanting to play Mario Kart or something.

Night.

Friday, March 6, 2009

CLS Compliant Code

How important is it to make certain your .NET code is CLS Compliant?

For the uninformed, CLS, or Common Language Specification, refers to the claim that CLS rules and restrictions are being followed.  However, the concept has a more specific meaning depending on whether you are describing CLS-compliant code or CLS-Compliant development tools, such as a compiler.  CLS-compliant tools can help you write CLS-compliant code.

FTA Writing CLS-Compliant Code [msdn]

I have personally been taking modest steps to ensure that my code is being validated CLS compliant for some time.  I do this almost automatically.

In the real world however, how important is this really?

Technically speaking, I believe there are a lot of reasons you should be concerning yourself with this.  But realistically, is it really that important?

Say you’re one of the, I don't know, millions of developers out there working with Visual Basic.  And like most .NET developers, sooner or later you are going to consume a class library that another developer wrote.  What if the Class Library was written in another language?

Lets not forget, this is .NET, and the language used shouldn't really matter.  But let’s say the library was written in C#. and a class was defined like this.

public class TestClass : Object {
public void doSomething() {
return;
}

public void DoSomething() {
return;
}
}




Yeah, I know, this example is stupid.  But bear with me, this is an important topic.  Simplicity makes this easier to understand.



VB Developers, this is allowed in C#.  But how would VB understand this?  After all, VB is NOT case sensitive, and it would get confused.  So how could VB possibly consume this without some sort of ambiguity?



The answer is, it simply cannot.  This is where the CLS comes in.  While forcing a validation does not DO anything to your code, it will prevent you from doing ‘stupid’ things.  Lets tag up that code a little.



[CLSCompliant(true)]
public class TestClass : Object {
public void doSomething() {
return;
}

public void DoSomething() {
return;
}
}


And how does Visual Studio take this?


clscompliantfail 



Note this does not prevent the code from building, But it is telling you that the the class fails CLS compliance.  Meaning, this would probably not work against all languages.



You could however expect normal operation if it were consumed by a C# project.



This is really as hard as it gets to validate your code.



In order to use this validation in your own code, you need to be sure to mark the assembly as CLSCompliant.  This is normally done in the AssemblyInfo.cs (replace with .vb or whatever).  Note to mark an assembly, the attribute is a little different,



[assembly: System.CLSCompliant(true)]




You do not need to specify the System namespace, but the AssemblyInfo.cs does not reference the System namespace by default.  So, pick your poison, import the namespace, or specify the namespace.



So, this was just something I wanted to share, I personally find it useful, but I welcome your thoughts.



On one more side note, I would like to mention, that this validation is only really concerned with aspects of your code that can be consumed by other assemblies.  This means that if one or both of these two methods were private, it would pass validation, no problem-O.

Thursday, March 5, 2009

Just wasting everyone's time

It seems like I’ve been to busy or lazy to manage this blog lately.  Not intentional, just to lazy.  So I figured I would waist your time, and just share something meaningless.

Any of you have a ‘favorite’ new feature of C#? or .NET in general?  They pushed a fair amount into the last release.

Personally I think the smallest, least important language updates are my favorite ones.  Two of which:

Automatic Properties is one of them.  Take the following code.

public class TestClass : Object {   
private int m_intValue;

public int Value {
get {
return m_intValue;
}
set {
m_intValue = value;
}
}
}




using automatic properties, you could shorten that to this.








public class TestClass : Object {   
public int Value { get; set; }
}



The end result is a virtually identical IL.  So.. there is no performance hit, the code is cleaner and easier to read and maintain, and as an added bonus, it saves typing.



Another update I thoroughly enjoy is the new method of assigning values/references to object properties, upon initialization.  Take this example:




TestClass cls = new TestClass();   
cls.Value = 55;





You could replace that with this:



TestClass cls = new TestClass() { Value = 55 };





In that example, perhaps it doesn’t look like it’s doing much, and perhaps you’re right.  But you aren’t limited to assigning just one property at a time, you can assign as many as you like.  And it’s still just considered one line of code, even if you split it across multiple lines, so if you’re the type who doesn’t like to use braces excessively, this can really help keep the code trimmed down.




foreach (String strPath in Paths)    
trv.Nodes.Add( new TreeNode() {
Text = strPath,
Tag = new FileInfo(strPath)
});



Anyways, I just wanted to share.  What are you guys’ favorite new features?

Tuesday, November 18, 2008

Super Crazy Linux Week

Well, not Super Crazy... but fun just the same.  Just this last week, I dug out my old laptop, and installed Linux on it.  Fedora Core 9 actually.  It's been fun...

Why would I do such a thing?  Well it started off as boredom,  I just wanted to.  Besides, the laptop wasn't being used for anything.

Once I got it set up, I started playing with mono.  For those of you who are still actually unaware of what mono is, it's an open source Microsoft .NET compliant framework.  Although my super technical definition may not be 100% accurate, you might want to look it up on wikipedia, or on the mono project homepage.

The last time I had played with mono was about 2 years ago.  And I have to tell you, I am very impressed with what I have seen this week.  I was able to compile a relatively complicated project built to run in Visual Studio 2008, for .NET 2.0.  While it did not execute exactly as expected, the areas it lacked were really not a big deal, and I was able to find ways around each and every issue I have faced so far.

My biggest complaint with Fedora Core, was the fact that the package manager did not have access to the most recent versions of the mono tools, and I had a great deal of difficulty compiling the latest to make it run on Fedora Core 9.  So my task now is to find the most perfect "out of the box" distribution for mono.

I installed Ubuntu to start my search.  The results weren't bad, but I totally missed the obvious.  I'm probably looking for OpenSUSE or SLES, since Novell is sponsoring the development of mono.  I'm burning OpenSUSE to DVD right now...

Anyways, out of the box, Ubuntu was a lot lighter weight, and more graphically appealing.  But how would it fare with immediate mono support.  Not bad actually.  I did have to add the 'universe' repository to the package manager, but I was able to get the most recent version of mono develop and a somewhat recent version of the framework up and running, and with just a few clicks of the mouse.

My beef with Ubuntu, is that the ASP.NET functionality did not work out of the box when I set it up in this way.  The updates also took significantly longer to apply with Ubuntu.

So tonight, I am going to try out OpenSUSE, and I'll post my results here.  If anyone has any opinion on the matter, or any good/bad experiences working with mono, I'd love to hear about it.

Saturday, November 8, 2008

"Traditional Sockets" in .NET -- Part 2

The other day I started talking about sockets in .NET. The whole reason this has been on my mind is because I have been discussing them with a coworker of mine. In Part 1 we created a simple server that would allow a telnet client to connect, the server would transmit a simple text message, and the server would terminate. Creating a simple client is not an awful lot different than creating a server. So I will just assume you have saved your Server project. If not, the code can be copy/pasted in part 1. Ok, you have your Server again? Great. We need to create a new project for the client. We can either do this in a new solution, in the usual way, or we can do this in the same solution. Keep in mind that if you choose to do your client in a new solution, any time I say run solution, I mean run the server, then the client. If you choose to add the client project to the current solution, Locate the solution node in the Solution Explorer Treeview Control, as seen here. The idea is that you need to right click this Solution Node to bring up a context menu, locate a sub menu called "Add", and in the sub menu select "Project". Creating this project is not an awful lot different than creating any other project. It will bring up a project template selection window, I would just select another Console application. The only trick once this project is selected, is to ensure that both the Client and the Server are starting at the same time. This can be accomplished by right clicking on that same Solution Node, to bring up a context menu, and by selecting properties on that context menu. This will bring up the solution Properties dialog. I have already set the properties properly. You want to chance the radio button selection from 'Single' or 'Current' to 'Multiple'. Then you want to choose 'Start' in both projects. Lastly, you want to make sure the Server is starting first. You can do this by selecting a project in the grid, and using the arrow buttons on the side. Once that is done, you can hit OK. Now we can move on making the client connect to the server. So make sure the source window for Program .cs in the Client project is open for editing. And make sure again that you are importing from the Networking Namespace and the Socket Namespace.
using System;
using System.Net;
using System.Net.Sockets;



Now move down to the Main function. I told you that we would define a socket in the client application in a very similar way. When we construct the socket, we are defining how the socket is going to behave, and this is highly important. So why don't we start with pretty well the same line of code, except that we will give it a more convincing name.

Socket socketClient =
new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
 

Looks similar right? That's good, because it should. Now if we think about the terminology, A Server Accepts connections from clients, whereas a client Connects to a server. You will notice that a member function for socketClient exists called Connect. Connect


is a little more complicated than Accept, in that it actually takes a parameter. You will notice that the parameter it expects is an EndPoint. Do you remember that an EndPoint on the server was basically an IP address and a Port? I hope your connecting the dots. What do you suppose would happen if we supplied an endpoint, that described a location where the server was listening for connections. I believe we specified the last endpoint as..
IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13001);


Oh but wait. We had decided to use the IPAddress.Any constant. We actually need to give it an address. If I were to leave you ponder about this for a while, or to try and make an address fit in there, you might eventually come to realize that even though IP addresses have numbers in them, as we generally express them, they are actually strings. So are web addresses. The string notation, 192.168.1.1 is expressed that way for a lot of reasons. Most of which is very much out of scope of this tutorial. In .NET, an address is a long number. You cant just plug in 192.168.1.1. And of course we cannot plug in www.google.com. Instead we use a facility called DNS (Domain Name System.) And while an IP notated string does not need to be looked up, it can be translated, an address like www.google.com would certainly have to be looked up. Luckily in .NET we can use the very same facility to look up either.
Dns.GetHostEntry("127.0.0.1");
Dns.GetHostEntry(www.google.com);


Very straight forward right? Good. Now also keep in mind that a lookup can return more than one IP address, or none. It's a great idea to make sure you are testing the output from such a lookup. We will not. This example is so simple, that not much will go wrong, and I will get into more detail if someone really wants to know. So lets create an EndPoint that can locate the server.
IPEndPoint epRemote =
new IPEndPoint(Dns.GetHostEntry("127.0.0.1").AddressList[0], 13001);


I of course renamed the IPEndPoint to make a little more sense. But now we can complete our connection. Why don't you make your Main function look something like this.
static void Main(string[] args) {
IPEndPoint epRemote =
new IPEndPoint(Dns.GetHostEntry("127.0.0.1").AddressList[0], 13001);
Socket socketClient = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socketClient.Connect(epRemote);

return;
}


You could technically run this, and it would probably execute so quickly that you would have not much idea anything happened. If it worked correctly, both the server and the client consoles would open then close. Now I will interrupt here for a second. Because there is a possibility that this will not work. Especially if you are using Vista. Vista configurations that I have seen have IPV6 support enabled. And when we do a look up, the GetHostEntry will return multiple addresses. If the first address that is returned is IPV6, then you will get an error indicating that the address is incompatible with the protocol. If you get this message, you can use the immediate window to locate the proper index, or you can just keep incrementing the index until it works. If anyone has any issues with that, I will gladly provide more insight. Moving along. So why did we not see the text in the client? You might be thinking the window closed to fast to read it. That's not the only reason you didn't see anything. But for now, lets solve that problem. Place this snippit in the bottom of both the client and the server.
Console.WriteLine("Press  to terminate");
Console.ReadLine();

Your code in the server should look like this:

static void Main(string[] args) {
IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13001);

Socket socketServer = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socketServer.Bind(epLocal);
socketServer.Listen(4);
Socket socketClient = socketServer.Accept();

byte[] bytSend = System.Text.ASCIIEncoding.ASCII.GetBytes("Hello, World!");
socketClient.Send(bytSend, 0, bytSend.Length, SocketFlags.None);

Console.WriteLine("Press to terminate");
Console.ReadLine();

socketClient.Close();
socketServer.Close();
}


And your code in the client should look like this:

static void Main(string[] args) {
IPEndPoint epRemote = new IPEndPoint(Dns.GetHostEntry("127.0.0.1").AddressList[2], 13001);
Socket socketClient = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socketClient.Connect(epRemote);

Console.WriteLine("Press to terminate");
Console.ReadLine();

return;
}


Try running the solution again. Both consoles stay open, but you only see the Message indicating that you can hit the enter key to terminate right? So why is it that the Telnet window gets the message, and our client does not. We do know that the client is connecting, because if you never ran the client, the server window would stay black, and you would get no Press Enter Message. Then if you attempted to connect with telnet again, you would get the expected result. The answer is simpler than you might think, or maybe I'm not giving you enough credit. Sockets work very similarly to working with files, or working with standard input / output. The truth is the client has in fact received the data, we just haven't read it out of the buffers yet. Why not add a read and print statement to the client.
static void Main(string[] args) {
IPEndPoint epRemote = new IPEndPoint(Dns.GetHostEntry("127.0.0.1").AddressList[2], 13001);
Socket socketClient = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socketClient.Connect(epRemote);

byte[] bytRecv = new byte[13];
socketClient.Receive(bytRecv, 0, bytRecv.Length, SocketFlags.None);
Console.WriteLine(System.Text.ASCIIEncoding.ASCII.GetString(bytRecv, 0, bytRecv.Length));

Console.WriteLine("Press to terminate");
Console.ReadLine();

return;
}


Ok, there are a few things happening here. Again, I am making a byte array, and I am running a similar operation off of the socketClient as that operation that we ran against the server. This one is called receive. Pretty simple right? Then I am simply using the ASCII encoding to convert the binary data back into the string. Test it out! Look at that... it's magic. There is still a matter that we must discuss however, and it's very important. You see we created the byte array by allocating only 13 bytes of memory. This is not nearly sufficient for almost any application that we will write. So why did I do this? Why didn't I allocate larger? I could have allocated larger. That is not the issue. The issue lies with the read/recieve operation. You see much like the write operation, the read asks me to specify a byte array to read data into, an index to start reading from, and a size, or in this case length to read. I told it to consume the whole array. I knew the message "Hello, World!" was 13 bytes. Lets Experiment. Try setting the array size to 10, then run the application. Runs as expected, the client pulls only 10 of the 13 bytes and both ready to terminate. Now try to set it to 100. Then Run it. Does it run as expected? No, not exactly. I think what you will find is that you get the message, but you're getting a lot of extra 'nothings'. In this example, perhaps it's not much of a problem. As your programs get more complex, especially once you get into asynchronous sockets, this becomes a real issue. In some circumstances, your application could deadlock waiting for more data to be written to the buffer before ending the receive operation. Ok, I think we can leave this at that for now. A lot of ground has been covered in the last two posts. Maybe a good time for you to experiment and become comfortable with some of the concepts. Also, if you can try changing things, and seeing if you can break it, that would be another great experience. However, for those who want more... I also said I would get into how to enable to server to accept more clients. Probably the easiest way to do this with what we have is to use a loop that keeps executing the Accept statement. And I think for at least the purposes of explaining how simple Blocking Sockets are, and some of their shortcomings, that is exactly what we will do. Let's do something rash. Lets create an infinite loop that accepts connections and sends our welcome message. Yes, I realize this will cause unreachable code, and I'm fine with that for now. This is just to illustrate a point, so bear with me...
static void Main(string[] args) {
IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13001);

Socket socketServer = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socketServer.Bind(epLocal);
socketServer.Listen(4);

Socket socketClient;
for (; ; ) {
socketClient = socketServer.Accept();
byte[] bytSend = System.Text.ASCIIEncoding.ASCII.GetBytes("Hello, World!");
socketClient.Send(bytSend, 0, bytSend.Length, SocketFlags.None);
}

Console.WriteLine("Press to terminate");
Console.ReadLine();

socketServer.Close();
}


OK, we can see that the client that connects get's its message, and is given the option to hit enter and then terminate. But the server, still nothing. Lets try to connect to it with telnet now, without restarting the server. It works.. so how do we stop the server? I think that's a good question. I mean yes, we could just terminate the process, but how about something a little more solid/proper? I'm going to do something now that I didn't plan on doing right away, because I think it would be highly beneficial, again this will illustrate the strengths and weaknesses of Blocking Sockets vs. Asynchronous Sockets. Are you familiar with Asynchronous Programming? If you said no, don't fret. If you have programmed in almost any visual environment, you're more than likely more familiar with Asynchronous Programming than you even realize. Asynchronous Programming allows us to perform certain tasks. while our main thread continues executing. Or maybe more appropriately, it allows us to wait for certain things to happen while our main thread continues executing. The main thread would get interrupted when this task completes. If you picture in your mind, programming the result of clicking a button, you're in the right mindset, on how the code will look. If we replaced the Accept call with a BeginAccept() call, that would kick off an asynchronous Accept. This means that rather than wait for the connection on the current main thread, the server will wait for the connection, and let us know when a connection has been made. This means that the server will walk right over BeginAccept Call, and then display the message that states we can press the Enter Key and the server will terminate. (of course we no longer need the loop). Once a connection is made, our callback gets fired. This is a method we designate at the time we call BeginAccept(). That method being fired is our notification that the Accept operation has been completed. If we call EndAccept passing in our AsyncResult, we get our client socket just the same as though we had called Accept() straight up. And there is no reason why we cannot call BeginAccept() again. So clear out that Accept loop altogether. And create a new Method. This method will be used as a callback when the Accept operation completes.
static void OnAcceptComplete(IAsyncResult result) {
Socket socketServer = (Socket)result.AsyncState;
Socket socketClient = socketServer.EndAccept(result);
byte[] bytSend = System.Text.ASCIIEncoding.ASCII.GetBytes("Hello, World!");
socketClient.Send(bytSend, 0, bytSend.Length, SocketFlags.None);

socketServer.BeginAccept(OnAcceptComplete, socketServer);

return;
}


This is a special method signature. returning nothing, and taking only an IAsyncResult parameter, this is the signature that pretty well all Asynchronous Methods will expect as a callback. The first thing we are doing is extracting the server socket, which is expected to be the Asynchronous state. (I will explain that in a few minutes) Next, we are going to call EndAccept on the server socket. This returns the client socket just like calling Accept Directly, as well as cleaning up the Async Operation. Then we send the "Hello, World!" message the same way we did previously. And we attempt to wait for another connection asynchronously. Now in order to tie this altogether, we need to kick off the AsyncAccept pattern in the first place. This can be accomplished by replacing the original Accept with a Begin Accept. Make your server application look like this.
static void Main(string[] args) {
IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13001);

Socket socketServer = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
socketServer.Bind(epLocal);
socketServer.Listen(4);

socketServer.BeginAccept(OnAcceptComplete, socketServer);

Console.WriteLine("Press to terminate");
Console.ReadLine();

socketServer.Close();
}

static void OnAcceptComplete(IAsyncResult result) {
Socket socketServer = (Socket)result.AsyncState;
Socket socketClient = socketServer.EndAccept(result);
byte[] bytSend = System.Text.ASCIIEncoding.ASCII.GetBytes("Hello, World!");
socketClient.Send(bytSend, 0, bytSend.Length, SocketFlags.None);

socketServer.BeginAccept(OnAcceptComplete, socketServer);
return;
}


Try and run the solution again now. You should see everything executes as expected. The client gets its welcome message, and a chance to terminate. So does the server. But if you don't close the server, you can start connecting telnet clients to it, notice they work as expected as well? This is really only one advantage to Asynchronous sockets. I will leave you on that note, because I know a lot of these concepts must be hard to swallow all at once. So please, if you have any questions, or I'm not clear enough on something, let me know, and I'll try and straighten you out. I suspect that will be the case, because my two year old is helping me out right now.. =) Next lesson, I will try not to introduce much more, but rather re-enforce some of these concepts. After all, if you can grasp all of this so far in just two lessons, you're doing way better than I did. I figure once we get some of these concepts down, I'll remove all the blocking stuff, and show you pure Asynchronous sockets, and some of the patterns that I use. I'd also like to show you some of the .NET features that actually make sockets convenient.

Thursday, November 6, 2008

"Traditional Sockets" in .NET -- Part 1

The title may be a little misleading, because when I say "traditional sockets", I don't necessarily mean I stick to any sort of convention... A couple of days ago, a co-worker of mine and I were talking; and sockets came up as an alternative to newer possibly more mainstream .NET technologies. I was asked to explain and maybe even provide a sample to pose as a starting point to become acquainted with sockets and network programming.

I did, and I felt that there may be some poor sap willing to tread through this entry and possibly learn something. Though, I'll warn you, there are at least 100 other blog entries that will explain this more properly.

Still here?... OK, read on...

I have come to realize that I can write for ages on the topic, and While I would love to show some of the more advanced features, I'm just going to loose my audience. So I'm going to start off really slow here. If you have some experience on the subject, please provide feedback. I love discussing these things. I'll progressively move into some more interesting parts of the framework I have combined with sockets to make for some really interesting applications.

Is it alright for me to assume that for the most part, readers are at least at an intermediate level with C#? I don't really want to explain standard convention, or syntax. Though I will gladly respond to anyone who has any question, or doesn't understand something.

So let's start this tutorial series, by creating a simple Hello, World! server. One that will accept a connection from a client, print the Hello, World! message and disconnect that client. For starters, let's just do a simple blocking server. Once the client has been disconnected, we will let main() return, and the server application will exit. It might not be practical in the real world, but who cares? lets have some fun! I'll get to the good stuff... but you need a foundation first.

For starters, create a Console Application in Visual Studio. I'm using 2008 Professional, but I believe this code will work for 2003 - 2008. Someone correct me if I'm wrong, I don't have the ambition to go through and run tests for each environment.

So create your Console application and start editing the Program.cs file.

Make sure your importing out of the networking namespace.


using System;
using System.Net;
using System.Net.Sockets;

Now move down to the static void Main(string[] args) method.

Let's talk a moment about what the server is going to have to do to accept a connection.

First, we have to realize that the server is going to have to lay down the law in terms of how it is going to communicate. We can explore your options in more detail down the road, or possibly in a few more tutorial lessons, but for now, lets just say that you will almost always be using something like this. Especially while you are learning.


Socket socketServer = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);

There really isn't a difference between a client and a server socket. At least when we construct it. However you will want to ensure that a client socket is created with the same rules as the server socket. This specific configuration will work over the internet, maintains a constant connection and ensures that data that is transmitted will reach it's destination, and the destination will receive the data in the same order that you sent it. It's very reliable, and very similar to most internet applications that you use, so the performance hit will not likely be something you are aware of. Again, we can get more into this later. Let's move on.

So what is the difference between a client and a server configuration?

A server "accepts" connections, and a client generally "connects" to a server. This is a great terminology, as this is the same terminology you will see using the socket framework.

We will get into how to set up a client later, probably next lesson.

If you were to look at the socketServer members (probably by typing socketServer. and looking through the list of members using intellisense) you will notice an Accept method. What do you want to bet that this is the method that is used to accept connections.

Lets look at our main method now,

static void Main(string[] args)
{
Socket socketServer = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
Socket socketClient = socketServer.Accept();
}

I hope your following along, this is all technically correct, however incomplete. We have a socket and defined how it behaves. We are also attempting to accept a client connection. But we really havent given the socket context.
Think back to almost any network program you have used. Generally when we attempt to connect to a service of any sort with a client, we must specify either a hostname or IP address, as well as a port number.
If you're new to this it's concievable to think that a server running on this machine will be available to any client that can see it, by it's IP address and / or Host name, as long as that host name will resolve into an IP address, and you're probably right. But what if your machine had more than one Network interface card. One facing the internet, and the other facing a LAN. What if you didn't want your server to be accessable by the WAN. No.. lets take this a step further, the two network cards are two separate network interfaces. Each technically participates in a separate network. And this really says nothing in the way of which port the server is serving on.
Did you know that only one server can exist on any one end point? An end point in this context is an address and a port number. Any one Address of this type can have up to 65535 ports open.

Ok enough theory, we can get into the specifics later. And yes, we do have a way of just opening the server on all interfaces at the same time.
Before we can accept connections to a socket, the socket must be bound to an end point. If you attempted to run that code right now, you would get an exception indicating something similar.
The good news is that it is very simple to both define an bind to an end point. You just need to be certain you arent attempting to bind to an end point that has already been bound to.

Try this.


IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13001);

This defines an edpoint using a constant that specifies any interface on the machine, and port number 13001. I doubt you have any service using port 13001, but if you do, just change it to something differnet. Just keep it in the higher numbers and don't exceed 65535. (most well known and used port numbers are lower values). We can later talk about that, but if you googled well known services or something similar, you'd get a list of common port numbers.

By the way, you defined an end point, you you havent used it yet. technically, you bind with socketServer.Bind(epLocal);


static void Main(string[] args)
{
IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13001);

Socket socketServer = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
socketServer.Bind(epLocal);
socketServer.Listen(4);
Socket socketClient = socketServer.Accept();

socketClient.Close();
socketServer.Close();
}
I also stuck in a Listen(4) in there. If you weren't aware, this allows me to specify how many clients can remain in the connect Queue to be accepted. This is NOT a maximum number of connections, I'm afraid you would have to code something like that yourself. This is useful because it would automatically turn away clients if the server was overloaded with clients attempting to connect. We will get into this later too. Don't worry about it now...

This would work now. You can try it, but you won't see much yet. If you were to run this, a console application will start, with a black screen, and appear to do nothing. You might even get a message from whatever firewall software you are using, asking if you want to block the server or not. I added the Close() statements, just to remind us that when we no longer need these resources, we should be releasing them.

If you were to open a windows console, and type something like telnet 127.0.0.1 13001, you would see your server window close, and you would probably not see anything in the telnet session, because the server hasn't sent any data.

Now even though you cannot see an awful lot happening, there is a lot more happening than meets the eye. The Accept() statement is very powerful, and must be used with consideration. Accept() is considered a blocking statement. When you call it, the application will wait until a client connects. This particular use of sockets is called "Blocking Sockets".

Blocking Sockets are a great place to start, because you can write very simple networked applications. You don't need to worry about a lot going on. Once we bring this tutorial into more real world examples, I'll introduce Asynchronous Sockets. Don't worry about this for now.

So let's complete our server. Why don't we send a Hello, World! to the client.

After the Accept() call, why not add something like this.

byte[] bytSend =
System.Text.ASCIIEncoding.ASCII.GetBytes("Hello, World!");
socketClient.Send(bytSend, 0, bytSend.Length, SocketFlags.None);

If this looks confusing, don't let it. I have simply taken the binary for the string Hello, World! represented as ASCII text, and set it into a byte array. Sockets transmit data in binary form.

The next statement simply says send data out of the bytSend array, start at index 0, and transmit the the full length of the array. The third argument is actually size, and because each byte is 1 byte in size, it naturally translates to number of indexes to transmit. Moving along..
We are not going to get into SocketFlags right now.

your program should look like this now.


static void Main(string[] args)
{
IPEndPoint epLocal = new IPEndPoint(IPAddress.Any, 13005);

Socket socketServer = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
socketServer.Bind(epLocal);
socketServer.Listen(4);
Socket socketClient = socketServer.Accept();

byte[] bytSend =
System.Text.ASCIIEncoding.ASCII.GetBytes("Hello, World!");
socketClient.Send(bytSend, 0, bytSend.Length, SocketFlags.None);

socketClient.Close();
socketServer.Close();
}

Go ahead, give it a try. Execute the code and then connect with the telnet client.

The server terminates, but the client now get's it's Greeting.

Congratulations, You've just written your first server. It's a silly server that does not do anything really useful, but a server none the less. Tomorrow I will try and find some time to elaborate on something more practical.

Next tutorial, I will focus on the following.
  • Creating a client application that connects to this server
  • And we will give the server the ability to wait for additional clients after the first, so it does not terminate after sending the message to the client.
  • I might move into two way communication as well, It really depends on how the client aspect looks on paper. I don't want to overwhelm anyone.
Now, that siad, if anyone needs any more detail on anything I've talked about, or has any requests on... well.. anything that I may know, please ask away.

I apologize for the fact that this is moving slow, I have no idea how on the ball you guys are, I just don't want to leave anyone behind.

Thanks for reading!

Followers