Design and the Importance of.

It is very good practice to design any applications well and thouroughly before commencing work on them. Here is the process that I prefer to take (and have taken in all recent projects):

1. Harvest as many customer requirements as possible.
2. Try to understand the way that your customers think, and write a small document listing the requirements as you understand them, trying to fit them into what the customers are trying to tell you.
3. Hand the document to co-workers and benevolent customers, asking them for as much feedback as possible.
4. Think seriously about the feedback, do not be offended to matter how scathing some of it may be. Try to put as much feedback back into the document as possible.
5. Create a preliminary specification covering as many points in the requirements as possible. Leave placeholders where you are unsure.
6. Hand this document to the same people to be checked again. Repeat the process until you are satisfied.
7. Design a structural specification. Define how different modules in your application are going to plug in to each other, and how they will communicate.
8. Hand the structural specification to your co-workers (after first proof-reading it), and correct any fallacies that they may find.
9. Test your concepts. Write several small applications to test certain ideas in the structural specification where you do not have experience. DO NOT use any of this code in the final product.
10. Once you have a good handle on how the application is going to fit together, hand out different modules to the correct people (perhaps only yourself) and begin coding. Again, if you become unsure about certain aspects, write some more proof of concept applications. You really should document anything that you have been unsure about, and found the right answer for.
11. After that, it is up to the QA of your company and yourself to find any bugs. Document all bugfixes that you make. This will help prevent you from making the same mistakes in the future.
12. All bugs found should be properly documented. Something like Bugzilla is really good.

Remember that proper planning may make the project seem slow at first, but it will save a lot of money and time in the long run. It will also majorly enhance the quality of the end product. The proof of concept applications will also serve to improve your programming skills to no end, before you even begin work on the actual project. If proper planning is not undertaken from the start, the price will be paid. As the project become larger and more convoluted, you will spend most of your time maintaining it with very little time for adding more features, and working on new products.

In the past, I have taken what were the requirements at the time, with no regard to the future. I delved directly into coding without thinking about future expandibility. This is biting me right now as I struggle to add new protocols, new parameters and features without ruining the structure of the application. I am very fortunate that I am fixing more bugs than I am creating, but I am certainly spending far far far too much time on it.

Another note, if people say bad things about your product, they are probably right. Maybe not fully correct, but correct enough for you to rethink your strategy. Luckily most of my complaints come from co-workers, and not from the customer. But I do know that my unplanned project has been becoming more and more awkward to use. The new one that I am working on now has been designed with all my past mistakes in mind.

Ammendation to code in previous post

I recommend that when using hidden child and parent id's in nested tables, that they should be in the last column. This is because if they are not saved in the XML file, the .NET dataset will automatically add them into the last column. So putting them in the last column at the start will ensure that they are always in the same place.

On a second note, it is usually better practice to save the schema along with the XML file and to leave the child and parent id's intact in the XML file (i.e. do not set them their mapping types as hidden). However, in my place, the consumers of data are very likely to include non .NET platforms such as linux and embedded code (e.g. in this case the Atmel AVR).

My first intended use for this is to create XML files to store different phrases in different languages for a globalised application that I am working on at work. When I have perfected my method of internationalisation in .NET, I will probably write a little article about that and how to make a sample application.

Embedded Tables in XML

Recently, I've been working on the design of new project that will be making extensive use of XML to store data. The project is also going to have to be translated into Chinese and Thai to begin with.

What I like about XML is of course the ability to embed data within data within data ad infinum. Yesterday I started researching the ability of .NET to read and write XML, and I found out that the DataSet object has XMLWrite and XMLRead methods. This made me quite interested. Having never used the DataSet object I read Microsoft's documentation about it which was inconsistent and unclear but understandable.

At first I had a little trouble while trying to get tables to embed properly. I was setting up relationships using parent ID columns and it all looked fine displayed in a DataGrid, but when I saved it to XML, it did not embed the data (which in the end was only one extra line of code to do).

Anyhow, I came across the solution this morning, which was very simple: when creating the DataRelation object, simply set the Nested property to true. Anyway, I thought seeing as I wasted several hours trying to find the answer to this (cursing Google everytime it gave me irrelavent sites), I decided that it would be worthwhile to publish it on this blog. So here is the code (with comments):

      //Declarations
      DataSet myDataSet = new DataSet("root");
      DataTable Ptable = new DataTable("object");
      DataTable Ctable = new DataTable("item");

      //Set up parent table (object)
      Ptable.Columns.Add("ID");
      Ptable.Columns[0].ColumnMapping = MappingType.Hidden; //hidden because I do
      //not want the Parent and Child ID's to show up in the XML, making it messy
      Ptable.Columns.Add("objectdata1",typeof(string));
      Ptable.Columns.Add("objectdata2",typeof(string));

      //Set up child table (item)
      Ctable.Columns.Add("parentID");
      Ctable.Columns[0].ColumnMapping = MappingType.Hidden;
      Ctable.Columns.Add("Age",typeof(int));

      myDataSet.Tables.Add(Ptable);
      myDataSet.Tables.Add(Ctable);

      // set up relation (Relation Name, Parent Column, Child Column)
      DataRelation myRelation = new DataRelation("objectitem",Ptable.Columns[0],Ctable.Columns[0]);
      myRelation.Nested = true; //this makes the tables embedded when written to XML
      myDataSet.Relations.Add(myRelation);

      //Add some meaningless data
      for (int i = 0; i < 4; i++)
      {
        DataRow parentTableRow = Ptable.NewRow();
        parentTableRow[0] = i; // ID to be referenced by the Child
        parentTableRow[1] = "some data " + i.ToString();
        parentTableRow[2] = "more data " + i.ToString();
        Ptable.Rows.Add(parentTableRow);

        for (int j = 0; j < 7; j++)
        {
          DataRow childTableRow = Ctable.NewRow();
          childTableRow[0] = parentTableRow[0]; //ID of the Parent row
          // I like using indexes because sometimes I change column names, but I
          // never change the column index.
          childTableRow[1] = j;  
          Ctable.Rows.Add(childTableRow);
        }
      }
      
      myDataSet.WriteXml("d:\\documents and settings\\vincent\\desktop\\childparent.xml");

The output is the following:

<?xml version="1.0" standalone="yes"?>
<root>
  <object>
    <objectdata1>some data 0</objectdata1>
    <objectdata2>more data 0</objectdata2>
    <item>
      <Age>0</Age>
    </item>
    <item>
      <Age>1</Age>
    </item>
    <item>
      <Age>2</Age>
    </item>
    <item>
      <Age>3</Age>
    </item>
    <item>
      <Age>4</Age>
    </item>
    <item>
      <Age>5</Age>
    </item>
    <item>
      <Age>6</Age>
    </item>
  </object>
  <object>
    <objectdata1>some data 1</objectdata1>
    <objectdata2>more data 1</objectdata2>
    <item>
      <Age>0</Age>
    </item>
    <item>
      <Age>1</Age>
    </item>
    <item>
      <Age>2</Age>
    </item>
    <item>
      <Age>3</Age>
    </item>
    <item>
      <Age>4</Age>
    </item>
    <item>
      <Age>5</Age>
    </item>
    <item>
      <Age>6</Age>
    </item>
  </object>
  <object>
    <objectdata1>some data 2</objectdata1>
    <objectdata2>more data 2</objectdata2>
    <item>
      <Age>0</Age>
    </item>
    <item>
      <Age>1</Age>
    </item>
    <item>
      <Age>2</Age>
    </item>
    <item>
      <Age>3</Age>
    </item>
    <item>
      <Age>4</Age>
    </item>
    <item>
      <Age>5</Age>
    </item>
    <item>
      <Age>6</Age>
    </item>
  </object>
  <object>
    <objectdata1>some data 3</objectdata1>
    <objectdata2>more data 3</objectdata2>
    <item>
      <Age>0</Age>
    </item>
    <item>
      <Age>1</Age>
    </item>
    <item>
      <Age>2</Age>
    </item>
    <item>
      <Age>3</Age>
    </item>
    <item>
      <Age>4</Age>
    </item>
    <item>
      <Age>5</Age>
    </item>
    <item>
      <Age>6</Age>
    </item>
  </object>
</root>

, which is exactly what I intended. This file can then be read directly into a DataSet, and it will set up the two tables as before, without the need for a schema or for the parent and child id's to be saved. It also makes the XML look a whole lot prettier.

In order to leave the child and/or parent ID's in the XML, set the ColumnMapping properties to MappingType.Element. When that is done, it is possible to seperate the items into a separate area within the XML by changing the Nested attribute on myRelation to false. For this to load correctly back into a DataSet object, the schema will need to be saved by adding the argument "XmlWriteMode.WriteSchema" to "myDataSet.WriteXML".