wxSQList2++ history/overview

Flowchart Summary of TIL Lab Core Processing Group

The previous project wxSQList++ was a complicated attempt to incorporate agreat deal of functionality into a single application.  This was a resounding success as far a demonstration of the procedures a logic to successfully export TheList.xlsb coherently to a local database, but the project was too bulky and too slow to be practical.

The second trimmed down iteration of the project, wxSQList2++, was built from some of the nicer portions of the original.  The scope of the application was simply to extract, analyze, and transform the data from TheList.xlsb.  This application only needed to access TheList.xlsb through use of the M$ Component Object Model (specifically OLE Automation of Excel), and output plain text to be utilized later.  Although this approach seems clunky, it executes quickly and provides data backup in a single step.

The output format was similar to that produced by mysqldump (dev.mysql.com/doc/refman/5.1/en/mysqldump.html).  This was done because at the time I needed a way of making quick snapshots of the data in the inventory, and I was also importing that data into two different databases (one local, one remote).  The output text contains all the commands and data required to rebuild the TIL Lab Inventory in a MySQL database schema.

The remaining functionality of the original wxSQList++ was augmented and included in a batchfile.  The batch file writes a log of the wxSQList2 application debugging output, copies the mysql compatible dump text of TheList.xlsb to an archive on the server, and finally invokes the mysql client in order to execute the sql dump as an import.  This system is mostly plain text and does not require messing with the C++ source code in order to change the archive locations or the database connection information.

Altogether, wxSQList2++, mysql client, and a batchfile take care of the “TheLIST to DB Export procedures” component of the flow chart.

Project: Wunderlist/SQList

Change of project name and method:

It has been quite a while since I last decided to put something down about the projects I have been working on.  The Wunderlist, which was a java & SQL system, has been put on hold indefinately.  While it would have been easier to do the migration of our labs data to Biofortis’ Labmatrix system from a java base, it was impractical as an intermediate step.  As such, the goal of the project has been preserved, but the implementation has been changed.  The new project is SQList.

The SQList in the current incarnation is a mySQL database that is a copy of the data contained in the monsterous Excel spreadsheet, “The LIST.”  The data is migrated to the database by a C/C++ application titled, “wxSQList++.”  This system is currently used solely by myself, but will soon be migrated off my local machine to a space hosted by the NCIweb group.

The devil and some details of the mySQL schema that is used:

• Schema’s name is currently “lab.”
  I intend on changing this to something such as TILLL (TIL Lab List), TheList, wList, or CG9.  It occurs to me that the naming really does not matter as any name choosen will not retain the original meaning beyond a year or two.  After that, the name will simply be a name.
• The Schema tables are:
  norminv:  Listing of the well formed biomaterial entries.  This holds the majority of the data.
  incoming: A copy of the entries being held in the Incoming sheet of The List.  These entries are manually examine prior to being merged with the main body of the normal inventory.
  abinv: Listing of the entries from The List that could not be parsed well or attributed to a subject.
  patients:  Listing of our subjects.  Having this table removes the need to repeat this information for each biomaterial entry that is attributed to a subject.
  controlsubjects: Listing of subjects that have materials that have been selected to be used as controls for assays (ie: Coculture IFNγ Release ELISA)
• Data Source:
  The wxSQList++ application is an interim tool that drops, creates, repopulates, and cleans all the tables in the schema from the information currently held in The List.  As such, the database is a snap shot of the inventory status and not a real time representation.

A bit about the wxSQList++ application:

This is a text only tool written in C\C++ that is designed for my use only.  I intend to replace it with a much more robust read and backup tool.  It’s purpose is to read the data out of The List excel spreadsheets and create a database representation of the information.

Projects List

• WunderList

  The first object of this project is to Importing patient and biological sample data out of a massive and irregular spreadsheet.  Efficiently parsing 66k rows by 30 columns into a more representational model is a huge undertaking, because I will also have to provide an interface for data input and queries.

  The second sub project is to develop an efficient user interface(s) for daily data input and useful queries.  This needs to be intuitive to the users of the current system, so the useful parts of the look and feel of the old interface (Exhell) will likely be mimicked.

• Predictive Artificial Neural Network
  

  Designing the middle layers of an ANN intelligently would be a futile task.  The greatest minds in immunology haven’t quite figured out which combinations of patient attributes will be predictive of response to treatment.  As such, the alternative to intelligent design is using an evolutionary process for arriving at a useful predictive model.

  This project will require methods for: collecting variables about patient & treatment, generating middle layer functions, generating weighting middle layer, model summation by thresholding, feedback to thresholding evaluation layer, model accuracy vs. training data.

• Cryogenic Preservation of Human Lymphocytes

  Examination of various cryoprotectants and freezing processes on both transduced and untransduced lymphocytes from PBMC.  These experiments are relatively straight forward.  Count cells before preservation, and then afterwards.  However, there are many nuances to this simple method to consider:  counting method, number of counts, when to count, standards between experiments, analysis of data, and so on.