Introducing Code Hinting for Regular Expressions

It's been a long time in the making but code hinting is here, and I'm pretty excited about it.



I've created a short code hinting demo that shows it off a bit. This feature is available only to licensed users. If you're a licensed user and you open up the tester now you'll see hints appear as you start typing an open parenthesis "(", bracket "[" or brace "{".

I took a lot of inspiration from Intellisense. In particular, I based my design off of the Intellisense in Expression Blend. I like the look & feel of Expression Blend. But the regular expression language is so different than anything Intellisense is used for. So functionality-wise I was on my own.

I knew one thing I didn't want. I didn't want the code hinting to aggressively autocomplete. In other words, Intellisense in Visual Studio will sometimes autocomplete whatever you're typing at the time, sometimes adding code that you don't even want. That's a side effect of their complex algorithm that's normally pretty good. However, the regular expression syntax is tricky. And I have this theory. Most people can't write regular expressions as quickly and easily as .NET. So attempting aggressive auto-completion doesn't make much sense when the user may not know what they're looking for until they see it in the description. I fear that if Regex Hero tries too hard to predict what you're going to type before you type it then it'll inevitably make mistakes.

So I've made it impossible for the code hinting to do anything you don't want it to. You literally have to select the pattern from the list to add it to your regular expression. You can do that either by using the arrow keys to move to it and hitting enter or tab to select it, or by double clicking the pattern. I think this still makes the code hinting easy to use, but without all the fuss of accidental code modification. In my humble opinion, the end result is pretty awesome. It's a beautiful addition that allows you to be even more productive. And if it saves you from having to look up something in the reference then I've done my job.

I still want to add a few patterns to the mix and make it work with replacement patterns as well. But for now I think it's ready for primetime and I'd love to hear your thoughts.

Proper Off-Site Backups Have Been Configured

When the site was hosted with Orcs Web they would backup the server for me as a part of their managed hosting. Now that I'm with Liquid Web and I've opted not to use their $100/month Guardian backup service, backups are on me.

So today I set up a database backup maintenance plan. Every night the server will create full backups for every database in SQL and save the backups to the extra (backup) hard drive in the server. I've got the backups configured to expire after 14 days. Essentially what this means is that I'll have quick restore capability for any day up to 2 weeks in the past.

But that's really not enough. So in addition to this I've signed up for MozyPro off-site backup. I've used MozyHome on my home computer for the past year and I've been pleased with their service. The MozyPro service is slightly more expensive than MozyHome. But MozyHome will not run on a server, so MozyPro it is.

I configured MozyPro to backup nightly just after the database backups occur. So every night the latest database backup files will be uploaded to Mozy's servers. In addition to that, the files that make up the websites themselves are also backed up to Mozy. The cool thing about the MozyPro service is automatic file versioning. They'll actually store 30 days worth of versions of every file that's backed up.

There are a bunch of ways one could go with backups. I came up with this scheme because it's simple, cheap, easily recoverable, and highly redundant. It was easy to set up too. I did all of this in about an hour.

Code Hinting for Regular Expressions : The Theory

I haven't heard much feedback on my code hinting post in January, perhaps because the first demo is not too impressive. And in fact I haven't been able to work on it as much as I wanted. But I'm still thinking about it and planning the user interface. There's still a lot to do to make this as well polished as Intellisense is in Visual Studio or Expression Blend. In fact I'm drawing more inspiration from the Expression Blend interface, but that's another story.

When I first started on this I knew that I needed to parse the regular expression on the fly and display the hints based on the text next to the caret. I begun by researching parsing theory. Now, I know what you're thinking. Why bother? Why not just use regular expressions? Well, I knew that regular expressions are ill-suited to parse HTML, and I thought that they may not be the best choice for parsing themselves, either. The reason I thought this, it turns out, was completely valid. Regular expressions are designed to parse regular languages. The regular expression syntax, however, is a context-sensitive language as listed in Chomsky's hierarchy. Beyond that I was somewhat lost when reading the complex labyrinth of articles on Wikipedia related to parsing theory, seemingly requiring a PhD to comprehend. But I was now equipped with some crucial knowledge with which I could write a parsing algorithm.

As I've often done, I wrote this algorithm completely from scratch so that I can understand it completely. After I wrote it and verified that it worked I determined that what I came up with is an embedded pushdown automaton capable of parsing mildly context sensitive languages. Perfect.

Essentially what this comes down to is that my algorithm is sophisticated enough to show the proper hints in the proper contexts. For example if you're starting a regular expression with a beginning parenthesis "(" then I should show a list of every possible construct that begins with a parenthesis: groups, lookarounds, comments, etc. But a beginning parenthesis "(" does not have the same meaning inside a bracket expression "[]". That's because a bracket expression follows its own rules. Secondly my algorithm is forward-looking and holds a memory as it parses from left-to-right, making it obscenely efficient for this task. Even with a huge 500+ character regular expression it'll know exactly what context the regular expression is in wherever the caret is at the time. And it'll do this in under 10 ms, faster than the human eye could even see it. This is almost too fast for your own good, but that's OK. I'm encouraged by how well it's working more than ever.

Stay tuned for updates.

UPDATE: Code hinting has been released!

Silverlight 4 Coming in April, or Maybe Sooner

The exact release date has not been announced. But Visual Studio 2010 RTM is coming out in April and I think it's safe to assume that Silverlight 4 will be released no later than that.

Each release of Silverlight has brought massive improvements over the previous version. And once again, Silverlight 4 does not disappoint. There is a long list of improvements but the ones that I think that will affect Regex Hero are as follows:

RichTextBox
My plan is to use this in place of all 4 major textboxes in Regex Hero. The new RichTextBox has built-in multiple undos & redos, so I can ditch my home-brewed code. It should be nice to use for syntax highlighting for the regular expressions I intend to create. It also has a built-in API to determine the pixel position of the text. I should be able to use this API and build a new highlighting scheme based off of it. This should do a couple things. First, I should be able to finally fix the problem I had with the ScrollViewer and ridiculously wide sections of text not scrolling properly with the highlighting. Secondly, I believe I'll finally be able to enable word wrapping and still have working highlighting.

Right-click support and custom context menus
Finally, I'll be able to create context menus with Cut, Copy, & Paste for the textboxes in the application.

Mouse wheel support
Oh yes, you'll be able to use the mouse wheel to scroll the text boxes.

Improved XAML parser
This should improve the start-up time of Regex Hero, if only marginally.

Like I said, there's much more in this release but these are the things I'm looking forward to most.

C# Code Generation Bug Fix and Improved "Jump To" Functionality

There was a bug in the C# code generation involving improperly escaped quotes. I'm using the string literal syntax with the @ sign. And when you do that, quotes are no longer escaped with \". Instead, it works more like VB with double quotes "" where the first quote escapes the second. I overlooked this but a fellow user of Regex Hero pointed this out to me today (thanks Jason). And after a small code change it works properly now.

Secondly, I made a small improvement to the universal match count / group count / stepper control in Regex Hero. The up/down buttons will normally select a match and scroll to it. But when there's only 1 match, then the up/down buttons are disabled. So in the event that there's only 1 match you can now click the text that says "1 match" and that will select and scroll to the match. It's a simple change but I think it helps.