Figure 1: Plot of normalized Area% deviation for two different baseline algorithms that are included in Virscidian’s Analytical Studio Professional – Compund QC product. The key baseline of interest is the AsLS2 baseliner.

The supporting detail

Sometime back we promised to explore more on quality of results processing in our blog post “Does the physical task of having someone review the results of purity and concentration output for substance confirmation make a difference?”. As we can all see time passed, lots due to project, travel and other commitments, but mostly because to do the type of analysis we had planned was either going to be a bear of a manual project or required us to develop some tools to help us do the job properly. Needless to say we went the second route, but this we had to put a little on the back burner whilst we attended to other pressing business needs. While I might be prepared to delve hour after hour into the depths of numbers and manual data review {and have done on many occasions}, we have better things to do and thus waited until we could automate much of the data gathering.

So armed with an experimental implementation of a data collection package, which we have embedded into our Analytical Studio Professional – Compound QC and Analytical Studio Reviewer, we set about trying to quantify the performance of some aspects of our software, which we share here as a forerunner of more detailed studies we intend on performing over time and publishing here and in other appropriate venues.

Customers have told us anecdotally that “they are extremely happy and impressed with the accuracy of the results we produce for our Compound QC substance analysis” by LC/MS/UV/ELSD, CLND (or CAD), which is all well and good but is entirely unquantified.

Experimental summary

For Compound QC or substance analysis in support of library management or Medicinal Chemistry – synthetic chemists or purification of crude substance, the calculated Area% of the target substance is used as a powerful indicator of the implied purity of the substance using some chosen detector and some extracted channel of data for the 3D type detectors that are available (DAD/PDA or LC/MS). In this case we chose the DAD 310 nm extracted wavelength chromatogram.

These results are based on a pilot small scale study of 70 samples of varying complexity from relatively clean to crude mixtures with lots of closely eluting chromatographic peaks, which we wanted to share and get feedback on. The final results are based on 52 extracted target substance peaks on a subset of the original 70 samples. This was because there were a number of samples where the target substance was not found or was identified as maybe.

The same data (UV310) were processed firstly with the Peak Picker baseliner, then the AsLS2 baseliner and then finally all results were reviewed manually and reintegrated to produce our gold standard result. The peak area% of the target substance for each sample was then extracted from each of the three series of results. The Area% values for the peak picker algorithm were then differenced from the manually reviewed results, as were the AsLS2 results. The two final sets of data were then plotted as we see in figure 1, though for ease of visualization the outlier point of 21.9% deviation for the Peak Picker algorithm was excluded from display, so that the fine detail of the graph could be observed. No other changes were made to how the data were processed or reviewed. Nothing fancy in the overall treatment and presentation of the data and results.

Results

We can see that the maximum deviation from our graph above is 0.5% for the AsLS2 baseline and was actually 21.9% for one data point using the Peak Picker algorithm {though this was excluded from the graph above}. We can also observe that there is a much tighter cluster for the AsLS2 algorithm than for the Peak Picker as shown below in Table 1.

Table 1:- Summary of results for the comparison of AsLS2 and Peak Picker algorithm against a series of manually reviewed gold standard results

Figure 2:- Summary of results for the comparison of AsLS2 and Peak Picker algorithm against a manually reviewed gold standard results

Conclusions

Using a development statistical data gathering approach, it is now possible to capture important data features, which can then be used to design performance evaluation studies within Analytical Studio Professional – Compound QC and Analytical Studio Reviewer, and as a consequence we are now able to start quantifying the performance of our software and further allow us to quantify improvements we make to our processing algorithms.

We hope you find this short investigation interesting and welcome feedback on our results, approach and new experiments that would further help quantify the performance of our software.