A smarter boot file for Simple DNS Plus v. 5

Several new features in Simple DNS Plus v. 5 require a new way to store the DNS zone list on disk.

Previous versions use a standard DNS "boot" file – a plain text file listing each zone line by line.
This format is very simple for anyone to read and update, which is why we have stuck with it this long.
Version 5 however needs to store additional information for each zone and update this much more frequently, making a different storage form necessary.

All the data from the "boot" file is actually cached in memory while Simple DNS Plus is running, so theoretically we could just read in all the data at startup, update the data in memory while running, and then save it all back to a file at shutdown. Only problem is if the computer shuts down unexpectedly (looses power), then the latest changes would not be recorded, leaving a mess.
So we need an efficient way to keep a continuously synchronized copy of the data on disk.

Using a full scale database system such as MySQL or Microsoft SQL Server would be overkill for this.

I spent a few days evaluating lightweight embedded database systems (see previous blog entry), but ultimately decided that this was too slow and added too much overhead and complexity as well.

The solution I eventually came up with was a simple proprietary file structure specifically shaped around the unique data of the DNS zone list with a few design ideas borrowed from xBase.
We’ll call it the "Simple DNS Plus Zone Database" file format – or ".sdpzdb" files.
Simple DNS Plus will store the file position of each record (zone item) in memory eliminating the need for database indexes, and making updates/deletes extremely efficient.

While this provides a lot of benefits, there is one major downside; since the new file format is binary, it can’t just be opened and edited with notepad like the old "boot" file.
We believe that was an important "feature", and so we will do what we can to make it easy to work with the new format as well.

For starters we have put together a small "Zone Database Viewer" application to quickly browse the data.
This can also export the data to the old "boot" file format or a CSV file.

And we plan on providing a .NET library to access the file as a standard "system.data.DataTable" object.

One remaining issue we have to deal with is concurrent updates to the database from different processes. Both the GUI record editor module (editrecs.exe) and the main server module (sdnsmain.exe) can update the zone list – potentially at the same time.
We can solve this either by using file locks, or possibly having all updates go through the main server module. The later would turn Simple DNS Plus into a simple database server, which may open up some other interesting possibilities…
We haven’t decided which way to go just yet.