Refactored Musings: August 2012

SVN With Visual Studio Express

I was looking for an integrated version control system (VCS) for my development environment. For my needs, it had to store revisions in the cloud to easily allow me to work from multiple computers and be simple to use. For this project, my IDE is Visual C# Express 2010 and we'll be using Google Code with Subversion (SVN) as the VCS with the TortoiseSVN client.

Google Code

I chose Google Code as my hosted VCS solution for a few reasons:

1) It encourages (requires?) its projects to be open source.
2) It uses my existing Google Account.
3) It allows multiple VCS implementations (Subversion, Git, and Mercurial).

Subversion

Subversion (SVN) is possibly the most popular VCS for code. It's based on a repository that contains the current revision of code, along with all previous revisions. To edit the code, a user checks out the latest version, updates it, and commits the changes back to the repository. This creates a new revision automatically. SVN keeps track of the revision of each file separately and also manages checkouts in a multiple user environment. Wikipedia has a good overview of SVN.

Visual C# Express 2010

Visual C# Express 2010 is a free IDE for the C# programming language. It is currently the latest version, but a new 2012 version has been announced for release later this year.

My language of choice is C# and I'm cheap, so this is the development IDE I prefer. Unfortunately, one of the main differences between Visual Studio and Visual Studio Express is that the Express editions don't support plug-ins. That rules out the possibility of an integrated VCS solution, but I continued to look for something that would simplify use of a VCS with Visual Studio Express.

(The setup instructions below do not use Visual Studio at all. I mention it here mostly to indicate why I chose TortoiseSVN as the SVN client.)

TortoiseSVN

TortoiseSVN is an SVN client for Windows that has shell integration (i.e. it integrates with the right-click menu in Windows Explorer). It will work with any SVN repository and is not tied to a specific programming language or IDE.

Setup

Now let's get everything set up. We'll assume that you already have the first revision of code created.

1) On the Google Code website, select "Create a new project". Enter the relevant information about the project and make sure to select Subversion as the version control system. Click "Create project" when completed.
2) Go to the project's website that Google Code has created. Go to the "Source" tab and leave this page up. We'll need some information on it later.
3) Download and install TortoiseSVN from the creator's website. The default options are fine for our install. After installation, restart the computer to enable TortoiseSVN's shell integration.
4) In Windows Explorer, right-click on the top-level folder for your code and select TortoiseSVN > Import. (I intuitively and mistakenly selected Export the first time. The Import and Export commands are named from the perspective of the hosted repository. In this case, the repository is importing the first revision of code.)
5) An import window will open up. Go back to the website we left open in step 2 and look for a line similar to the following: "svn checkout https://project-name.googlecode.com/svn/trunk/ project-name --username user@gmail.com". Copy the URL beginning with "https://" and ending with "trunk/" and paste it into the URL field from the import window. Click OK in the import window.
6) You will now be prompted for your username and password. Look at the website we used in steps 2 and 5 again. Click on the "googlecode.com password" link to go to a website that will show your SVN credentials. Note that this password is not the same as your Google Account password. Copy and paste this information into the import window's login screen. You can also select to have TortoiseSVN remember these credentials.

Usage

At this point, Google Code and TortoiseSVN are all setup with a copy of your code. Each time you want to edit the code, there is a simple 3 step procedure to follow:

1) Right-click in a local folder and select "SVN Checkout...". Confirm that the URL of repository is the same as we entered previously, and then click OK. This will create a new folder containing the latest revision of your code.
2) Using your editor or IDE of choice, edit the code.
3) When you're ready, right-click on the project folder again and select "SVN Commit...". The commit window will automatically select the files that have been updated. Click OK to begin the commit.

Now, head on over to GrokCode to learn how to use source control effectively.

Serial Communication in Linux

Today I set up serial communication with a linux server and thought I'd document the procedure here. The following instructions assume your linux distro uses GRUB. This was tested on CentOS 5.6 (ish) and Windows 7.

We'll refer to the Linux computer that we are trying to connect to as the "server". The computer we are using to make the serial connection will be the "client".

Procedure

Step 1) On the server, open /boot/grub/grub.conf in a text editor and find the entry for your current kernel. It will look something like this:

default=0
timeout=5
hiddenmenu
title CentOS (2.6.18-194.17.4.el5.centos.plus)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-194.17.4.el5.centos.plus ro root=/dev/VolGroup00/LogVol00 rhgb quiet
        initrd /initrd-2.6.18-194.17.4.el5.centos.plus.img

You may see multiple title sections. Each of these is another possible boot option in GRUB. Every time a new kernel is installed, a new boot entry is added to this list. The old boot entry and kernel are left behind so that you can boot with them if you end up having any issues with your system.

In order to have terminal output displayed over serial during bootup, we need to append some serial settings to the appropriate boot entry. Each title section is implicitly numbered starting with 0. Take a look at your default=# line to determine which entry is booting up by default and add your changes to that entry. Or you could just add the serial settings to every entry.

default=0
timeout=5
hiddenmenu
serial --unit=0 --speed=38400 --word=8 --parity=no --stop=1
terminal --timeout=30 serial console
title CentOS (2.6.18-194.17.4.el5.centos.plus)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-194.17.4.el5.centos.plus ro root=/dev/VolGroup00/LogVol00 rhgb quiet console=tty0 console=ttyS0,38400n8
        initrd /initrd-2.6.18-194.17.4.el5.centos.plus.img

Step 2) Open /etc/inittab in a text editor. Add the following two lines at the bottom of the file:

T0:23:respawn:/sbin/getty -L ttyS0 38400 vt100
T1:23:respawn:/sbin/getty -L ttyS1 38400 vt100

Step 3) Connect the server and client computers together with a serial cable.

Step 4) Open up a serial communication program on the client computer. I prefer to use Hyper Terminal. This program was included free in Windows XP, but is not part of Windows Vista or later. If you can copy the program from a Windows XP computer, it will still run in newer versions of Windows.

Step 5) Configure Hyper Terminal for to use the proper serial settings. For us, that will be:

Port: COM3
Bits per second: 38400
Data bits: 8
Parity: No
Stop Bits: 1
Flow Control: Hardware

If you are using an actual serial port on the client, you will most likely be using COM1 or COM2. If, like me, you are using a Serial-to-USB converter, it will probably be COM3. If the above settings still don't work, try changing Flow Control to None.

Final

At this point, you will be able to see the normal terminal output during server bootup on the client's serial display. Once the Linux server is booted up, you will be presented with a login prompt giving you shell access to the server.

Regular Expression Lookarounds

(This post deals with creating a single regular expression that matches a valid string syntax and then only matches a specific subset of those valid strings.)

I've been working with regular expressions on my current project. I've used them in the past but I bumped into something that stumped me with my current problem. I've got a list of file names that I'm checking for validity with our document control naming scheme. My current task is to look for any file names that have lowercase letters but are otherwise valid. These will then be flagged for review and then automatically fixed to uppercase letters.

My first step was to create a regular expression that matched valid file names:

^[0-9]{3}-[0-9]{4}-[0-9A-Za-z][0-9]{2}-[0-9a-zA-Z]{2}-[0-9A-Za-z]{1,3}

In a more human-readable form, this means that the file names must be of the form: ###-####-X##-XX-Y where # is any digit, X is any alphanumeric character, and Y is a section of alphanumeric characters of variable length between 1 and 3 characters. Example valid name: 123-4567-A89-B0-11

This regular expression worked out exactly as intended, but as part of this project I also learned some shorthand which simplifies the above expression and makes it more readable:

^\d{3}-\d{4}-[[:alnum:]]\d{2}-[[:alnum:]]{2}-[[:alnum:]]{1,3}

This takes advantage of two shorthand elements: \d is equivalent to [0-9] and [[:alnum:]] is equivalent to [0-9A-Z-a-z]. Unfortunately, C# doesn't seem to recognize the second of those two shortcuts. Gordon McKinney has a good reference sheet for regular expressions.

The next step was to look for file names that were matched by the previous regular expression but have a lowercase letter in them. The easy way to do this would be to save the list of matches from the first expression and then match against the following expression to look for any lowercase letters:

[a-z]

[[:lower:]]

Simple, right? Except that my requirements were to accomplish this all in one regular expression. I experimented with a few things before finally stumbling on lookarounds. I never could get my solution working with a positive lookahead (I think the problem was with the starting anchor), but I did get it working with a positive lookbehind:

^\d{3}-\d{4}-[[:alnum:]]\d{2}-[[:alnum:]]{2}-[[:alnum:]]{1,3}(?<=[[:lower:]])

This expression is first looking for the above-mentioned valid naming scheme, and then it looks back through the regular expression for anything matching [[:lower:]]. Upon finding a match, the entire expression is now considered matched.

UI Responsiveness

Today we're going to look at breaking out computationally-long tasks into separate threads to keep the UI of a program responsive while background processing is going on. Basically, we want to have one thread that focuses on handling the UI, and a separate thread for a long-running process.

Sample Program

I frequently write programs that asks the user for a bit of input, and then get bogged down processing that input for a significant amount of time. As an example, let's use the following C#/WPF program:

<Window x:Class="WpfApplication1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="350" Width="525">
    <DockPanel>
        <DockPanel DockPanel.Dock="Bottom">
            <Button x:Name="startButton" DockPanel.Dock="Right" Height="40" Width="80" Click="StartButton_Click">Start</Button>
            <TextBox x:Name="userInput" />
        </DockPanel>
        <TextBox x:Name="resultDisplay">No Results</TextBox>
    </DockPanel>
</Window>

public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
    }

    private void StartButton_Click(object sender, RoutedEventArgs e)
    {
        this.ProcessStuff(Convert.ToInt32(this.userInput.Text));
    }

    private void ProcessStuff(int maxCount)
    {
        double result = 0;

        for (int i = 0; i < maxCount; i++)
        {
            for (int j = 0; j < maxCount; j++)
            {
                result += i * j;
            }
        }

        this.resultDisplay.Text = result.ToString();
    }
}

This code will accept an integer as user input. Once the 'Start' button is clicked, it will do some arbitrary math processing on this integer (larger integers increase computation time), and will then output a result.

On my computer, an input of 10,000 or less will return a result less than a second after the 'Start' button is clicked. However, an input of 100,000 takes about a minute and during this time, the user interface is completely frozen. The program doesn't even redraw the window because the same program thread that handles updates to the window is busy in our nested for-loop. Until that loop completes, it can't take any time to update the display. This is a great way to get support calls from customers thinking that your program has frozen.

Making a Separate Thread

To fix this, we need to move the for-loops into a second thread so that they don't tie down the main thread. First, we'll have to add a new 'using' statement up at the top of our code that will give us access to the Threading namespace:

using System.Threading;

Next, we create a new thread to run the ProcessStuff function, and then we start this new thread:

private void StartButton_Click(object sender, RoutedEventArgs e)
{
    // Create a new thread for the ProcessStuff function.
    Thread t = new Thread(this.ProcessStuff);

    // Start the thread and pass in the user input as an object.
    t.Start(Convert.ToInt32(this.userInput.Text));
}

private void ProcessStuff(object maxCount)
{
    // The Thread object requires parameters to be passed as objects,
    // let's cast that back to an int.
    int max = (int)maxCount;
    double result = 0;

    for (int i = 0; i < max; i++)
    {
        for (int j = 0; j < max; j++)
        {
            result += i * j;
        }
    }

    this.resultDisplay.Text = result.ToString();
}

This code will keep the UI responsive after clicking the 'Start' button, even with a large number in the input field. At least, it will stay responsive right up until the unhandled exception. The last line in ProcessStuff will throw an InvalidOperationException stating that "The calling thread cannot access this object because a different thread owns it." We moved ProcessStuff into it's own thread, and that last line is trying to modify a WPF control back in the main thread. Modifying members that don't belong to the current thread doesn't work.

What we need is a way for the ProcessStuff thread to run code back in the main thread. This code can then perform the update for the ProcessStuff thread. To do this, we'll use delegates and the UI thread's Dispatcher.

Delegates

First, let's create a new method that will accept a result of type double and then display this result in our window:

private void UpdateResults(double result)
{
    this.resultDisplay.Text = result.ToString();
}

Next, we'll set up a delegate using this method. Delegates are data types that reference a method. Think of them as variables that hold methods instead of numbers or text. First, we need to define a new delegate type that matches our UpdateResults method signature. We'll add this as a class member in MainWindow:

// This needs to match the signature of ProcessStuff.
delegate void DisplayUpdater(double result);

Then, we'll use this delegate type in our ProcessStuff method:

private void ProcessStuff(object maxCount)
{
    // The Thread object requires parameters to be passed as objects,
    // let's cast that back to an int.
    int max = (int)maxCount;
    double result = 0;

    for (int i = 0; i < max; i++)
    {
        for (int j = 0; j < max; j++)
        {
            result += i * j;
        }
    }

    // Use the delegate type we defined to create a new DisplayUpdater
    // that holds a reference to the UpdateResults method.
    DisplayUpdater resultsUpdater = new DisplayUpdater(this.UpdateResults);
}

Dispatcher

Now it's time to introduce the Dispatcher. The Dispatcher is in charge of handling the work a thread needs to perform and therefore each thread can only have a single Dispatcher. When events are fired or layout changes are made, the Dispatcher puts each of these tasks in a queue and works its way through them all in order. In our original program, the Dispatcher couldn't start a drawing update until the ProcessStuff method had completed.

Now that we've got ProcessStuff in its own thread, we need to add a call to the UpdateResults method in the Dispatcher's queue. This is done with the BeginInvoke member of the Dispatcher which takes a delegate parameter. Every WPF control contains a reference to the Dispatcher of its current thread. So we can access the main thread's dispatcher using the control we wish to update, resultsDisplay. Now, our updated ProcessStuff is as follows:

private void ProcessStuff(object maxCount)
{
    // The Thread object requires parameters to be passed as objects,
    // let's cast that back to an int.
    int max = (int)maxCount;
    double result = 0;

    for (int i = 0; i < max; i++)
    {
        for (int j = 0; j < max; j++)
        {
            result += i * j;
        }
    }

    // Use the delegate type we defined to create a new DisplayUpdater
    // that holds a reference to the UpdateResults method.
    DisplayUpdater resultsUpdater = new DisplayUpdater(this.UpdateResults);
    
    // Add a call to resultsUpdater / UpdateResults() to the Dispatcher for
    // our resultDisplay TextBox.
    this.resultDisplay.Dispatcher.BeginInvoke(resultsUpdater, result);
}

Success

At this point, we've moved the time-consuming for-loop into a separate thread. Then we added a delegate reference to the UpdateResults method. This allowed us to queue up a call to that delegate on the main thread's Dispatcher. Now we can run the program with larger inputs and the UI will remain responsive. The trick to UI responsiveness is to make sure that all tasks in the Dispatcher's queue are short. That way UI update tasks will continue to be completed on time.

This responsiveness offers an immediate benefit because the user no longer thinks the program has frozen. However, we could also take advantage of this new functionality by showing a progress in some way or allowing the user to work on other aspects of the program.

Final Code

Here is the final C# code with the addition of a busy indication once the 'Start' button has been clicked:

namespace WpfApplication1
{
    using System;
    using System.Threading;
    using System.Windows;

    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        // This needs to match the signature of ProcessStuff.
        delegate void DisplayUpdater(double result);

        public MainWindow()
        {
            InitializeComponent();
        }

        private void StartButton_Click(object sender, RoutedEventArgs e)
        {
            // Create a new thread for the ProcessStuff function.
            Thread t = new Thread(this.ProcessStuff);

            // start the thread and pass in the user input as an object
            t.Start(Convert.ToInt32(this.userInput.Text));

            // Indicate that the program is busy with the current input.
            this.startButton.Content = "Working";
            this.startButton.IsEnabled = false;
        }

        private void ProcessStuff(object maxCount)
        {
            // The Thread object requires parameters to be passed as objects,
            // let's cast that back to an int.
            int max = (int)maxCount;
            double result = 0;

            for (int i = 0; i < max; i++)
            {
                for (int j = 0; j < max; j++)
                {
                    result += i * j;
                }
            }

            // Use the delegate type we defined to create a new DisplayUpdater
            // that holds a reference to the UpdateResults method.
            DisplayUpdater resultsUpdater;
            resultsUpdater = new DisplayUpdater(this.UpdateResults);
            
            // Add a call to resultsUpdater / UpdateResults() to the Dispatcher
            // for our resultDisplay TextBox.
            this.resultDisplay.Dispatcher.BeginInvoke(resultsUpdater, result);
        }

        private void UpdateResults(double result)
        {
            // Update the display with the given result.
            this.resultDisplay.Text = result.ToString();

            // Indicate that the program is ready for a new input.
            this.startButton.Content = "Start";
            this.startButton.IsEnabled = true;
        }
    }
}

Questions and comments are always welcome.

Setting up a Blogspot

I've decided to start a blog chronicling new programming techniques that I learn. I find that I frequently spend time learning the ins and outs of some aspect of programming, but then I end up not using it again for months. When I do need to use it again, I've forgotten most of what I've learned and end up digging through old code to use as an example as I relearn the technique.

This blog is designed to fix that in two ways:

1) I'll write a summary of what I've learned, including code samples, that I can easily refer back to at a later date.
2) The act of writing down the summary should further cement the concept in my brain, and maybe I won't forget it in the first place.

SyntaxHighlighter

That said, my first task I'll be writing about is setting up Blogger / Blogspot for code snippets and syntax highlighting. For this purpose, I chose to use SyntaxHighlighter, which is a JavaScript solution created by Alex Gorbatchev. There are other options out there, but SyntaxHighlighter is popular and easy to use. Some blog sites even include SyntaxHighlighter as a built-in option. Let's see what it takes to get set up with a simple SyntaxHighlighter tutorial.

Setup

First, you need to decide if you want to host the SyntaxHighlighter files yourself, or simply link to the hosted version on the creator's website. For Blogger, we can't upload JavaScript files (as far as I know), so we'll be linking to the hosted files. If you want to host the files yourself, simply download them and replace the URLs below with relative URLs to your file location.

Now, we need to set the page up for SyntaxHighlighter. We need to add a bit of code to the <body> section of our HTML (I suggest adding it right before the </body> closing tag):

<link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>
<script src="http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js" type="text/javascript"></script>
<script src="http://alexgorbatchev.com/pub/sh/current/scripts/shAutoloader.js" type="text/javascript"></script>
<script type="text/javascript">
//<![CDATA[
function path()
{
  var args = arguments,
      result = []
      ;
       
  for(var i = 0; i < args.length; i++)
      result.push(args[i].replace('@', 'http://alexgorbatchev.com/pub/sh/current/scripts/'));
       
  return result
};
 
SyntaxHighlighter.autoloader.apply(null, path(
  'applescript            @shBrushAppleScript.js',
  'actionscript3 as3      @shBrushAS3.js',
  'bash shell             @shBrushBash.js',
  'coldfusion cf          @shBrushColdFusion.js',
  'cpp c                  @shBrushCpp.js',
  'c# c-sharp csharp      @shBrushCSharp.js',
  'css                    @shBrushCss.js',
  'delphi pascal          @shBrushDelphi.js',
  'diff patch pas         @shBrushDiff.js',
  'erl erlang             @shBrushErlang.js',
  'groovy                 @shBrushGroovy.js',
  'java                   @shBrushJava.js',
  'jfx javafx             @shBrushJavaFX.js',
  'js jscript javascript  @shBrushJScript.js',
  'perl pl                @shBrushPerl.js',
  'php                    @shBrushPhp.js',
  'text plain             @shBrushPlain.js',
  'py python              @shBrushPython.js',
  'ruby rails ror rb      @shBrushRuby.js',
  'sass scss              @shBrushSass.js',
  'scala                  @shBrushScala.js',
  'sql                    @shBrushSql.js',
  'vb vbnet               @shBrushVb.js',
  'xml xhtml xslt html    @shBrushXml.js'
));
SyntaxHighlighter.config.bloggerMode = true;
SyntaxHighlighter.all();
//]]>
</script>

That uses the autoloader feature of SyntaxHighlighter 3. It will always load two core CSS files, the core javascript file, and the autoloader script, but it will only load specific brush scripts if the corresponding code language is used on the current site. If you opt for SyntaxHighlighter 2, you'll have load brushes even on pages that don't need them.

You'll also notice that bloggerMode is set to true. This is only necessary on Blogger and defaults to false for use on other sites.

Samples

Now you can start adding code snippets to your website. For example, to add the following C# code snippet:

public HelloWorld()
{
    Console.WriteLine("Hello, World!");
    this.Name = "blah";
}

Add the following to your HTML using the <pre> tag:

<pre class="brush: csharp; toolbar: false">
public HelloWorld()
{
    Console.WriteLine("Hello, World!");
}
</pre>

Or add the following code to your HTML using the <script> tag:

In both methods, the "class=" parameter tells SyntaxHighlighter which language to expect. You can also add other options here as described on SyntaxHighlighter's configuration page.

There are some differences between the two methods though. With the <pre> method, any HTML entities need to be encoded properly (e.g. '<' becomes '<'). This means that in order to show the following HTML snippet:

<body>
  <p>Hello, World!</p>
</body>

You need to have the following in your HTML file:

<pre class="brush: html; toolbar: false">
&lt;body&gt;
  &lt;p&gt;Hello, World!&lt;/p&gt;
&lt;/body&gt;
</pre>

The easiest way to convert your code samples is to use Quick Escape. This site allows you to paste in text and it then properly encodes any HTML entities for you to copy back to your website.

If you choose to use the <script> tag, you don't need to worry about HTML entities (the CDATA bit takes care of that), but there are some other downsides. The main issue is that the code in the <script> tag will get stripped out by some browsers or RSS readers. This includes the "Compose" feature of Blogger's post editor.

The <pre> tag will still format a bit like code even when <script> tags are removed or SyntaxHighlighter isn't working. Here's the C# sample with just the <pre> tag but no SyntaxHighlighter:

public HelloWorld()
{
    Console.WriteLine("Hello, World!");
}

You may have noticed that all of SyntaxHighlighter samples have line numbers except for the first. That's because the newest version of SyntaxHighlighter, 3.x, doesn't handle double-digit line numbers correctly. The solution is to either remove the line numbers all together or use an older version of SyntaxHighlighter. Remember though, if you drop back to SyntaxHighlighter 2.x, you lose the autoloader feature I mentioned earlier.

Optimization

I noticed in Chrome's Web Developer Tools that using the hosted SyntaxHighlighter files causes double requests to be issued to get each file. This is because the public address that SyntaxHighlighter recommends is at http://alexgorbatchev.com/pub/sh/current/, but the actual files reside at http://agorbatchev.typepad.com/pub/sh/3_0_83. Changing to the typepad address cut my page's load time down by about 30% from 1.1s to 0.76s. The downside to using the typepad address is twofold. First, my blog won't automatically be using the newest version when it is released. Second, if the files are ever moved to a new host, the redirection address that you're supposed to use will probably be updated, but the typepad address I'm using will go dead.

Acknowledgements

This post was put together using information from the following sites:

SyntaxHighlighter
Adding a Syntax Highlighter to your Blogger blog
Awesome code syntax highlighting made easy including a comment by "freak"

That's all for now. Please let me know in the comments if this has helped you or if you have further questions.