2.1.6.4 Risk Mitigation and Problem Resolution

Risk Management details how to manage the risks associated with a project. It details the risk management tasks that will be carried out, assigns responsibilities, and lists any additional resources required for the risk management activity. IBM Risk Mitigation Reference

Problem resolution describes the process used to report, analyze, and resolve problems that occur during the project. Our process is to report the problem in this document, resolve the problem through meetings, and document that resolution.

Among several risks that we have acknowledged as possible in the Lifecycle Objective Milestone, we've seen many already present themselves.

The first item in software risk evaluation is programmer hubris. The biggest battle is to convince the team that the problem is indeed formidable, that the obvious tricks aren't going to yield a quick solution, and that a lot of work is required between when software is initially functional and when it is market-ready. This classification problem is not simple, and no solution will be universal or optimal. It is imperative to acknowledge that iteration plans are important only because they organize a process that otherwise is easily derailed, and that there are good reasons why experts are still considered better at classifying cells, even when they can't explain their reasoning.

Using the Rational process is extremely useful in mitigating this problem. The recognition of an important Transition development phase after Construction, forces the schedule to accommodate user and customer feedback. The distinct requirement for milestones to be prescribed and then met, as well as the specific requirements that RUP lays out for documentation, mollify the risk of not meeting business needs.

Of the external risks, we've had two collaborators -- one an author of an algorithm, the other an author of a use case -- who have not responded to our contact attempts. To mitigate this risk we've sent letters explaining our reasons to discontinue our collaboration. This example highlights a risk of not being able to meet all of the prescribed goals but also shows how the project plan must remain malleable.

A repeated issue we've faced is the lack of organization in the plethora of documents required by the process. Often we have started a document, only to find that it exists already, in some iteration. We are mitigating this issue by keeping all relevant documents in one place, accessible by all. We have created a new taxonomy that emphasizes the project plan, as opposed to the grant application documents, and have retitled all documents according to their place in the taxonomy.

An issue we've encountered is the amount of time that has been spent on project management. We had anticipated a small amount of time spent on Project Management, with the majority of our efforts on the engineering of this project. We are solving this problem by creating a project plan that we can all reference going forward and have adjusted our estimates of the effort required by project management, an integral part of the project.

For a project our size, the value is that RUP supplies the vocabulary, schedule, and milestones. Accounting and task management software are overkill for a project that can fit its team in one room every week. The biggest lesson learned from this process is that OminiPlan project management software was an excessive burden to our small team and created odious roles for those managing the reporting. Several of the standard Rational mechanisms of Work Lists, Progress Monitoring, and Performance Monitoring have been excluded because they simply don't work without middle management to feed them. Project management has been reorganized to become leaner and to not gather data that isn't being tracked.

Task-based management has a tendency to encourage a depth-first traversal of the problem space. Whenever a problem is deemed complex, it is divided into subproblems and delegated in parts. The recursion of delegation ends up involving more people and generating tasks to scope out tasks, until the connection is lost between the specific work and the whole problem. Discipline needs to be exercised to keep problem solving in a breadth-first pattern and to not break down problems that can be solved as a whole, lest we run the risk of Dilbertizing the process.

Risk/Problem	Mitigation/Resolution
Unable to scale up Use Case, Algorithm, and Metric analysis without a working database and repository of files	Project Plan adjusted to include a DB and repository.
Algorithms may not work to select target populations	Algorithm improvement, combinations of algorithms, created modular framework that supports any external classification methods
Universe of metrics is not fixed. New publications or network contacts may add metrics.	New metrics can be added to the modular structure of FlowDx.
Match Ratio may not adequately describe the accuracy of one classification to another in context of biological or diagnostic decision.	Need to add another metric of diagnosis or biological answer to question being asked in Use cases.
“The software should recognize unexpected events. The software should identify unexpected events such as acute lymphoma in addition to the target of lymphoma.” Dr. Truc Pham at InCyte Pathology Associates	Create algorithm to recognize unexpected events as part of the requirements of FlowDx.
WEKA for neural net analysis: data handling was found to be tedious as WEKA requires CSV files for input, has no ability to batch any type of operation, and has a very limited size of input data.	ANN and SVM work will be done in MatLab and R.
Requirements of the project may not be fairly stable and well understood.	Define and communicate requirements through project plan.
Project Priority Issues. Focus on FlowJo releases by PI, Engineers, Application Scientists, and Project Manager, until release of 7.5	Priority of FlowJo Releases over FlowDx did not change.
Confusion regarding implementing the project plan using RUP.	Created the project plan as a group while citing RUP models that we can all follow.
Efforts and documents are being duplicated.	Started to use Google docs as the primary method of communication and document organization.
Project Plan requirements don't match grant requirements -- time is wasted creating two different docs.	Create Project Plan documents as first priority and more complete information, make grant document second, pare down to limited length, and add additional required pieces.
OmniPlan project management software is an excessive burden to our small team, creates odious roles for those managing the reporting. We have not found a good software solution to show tasks, track/display progress, and attach deliverables. Frustration at the focus on time rather than the deliverables of the tasks.	Want an ability to view the overall picture but also to drill down to tasks for each person and progress on the project. Have spent effort using OmniPlan, which was deemed too complicated. We've agreed upon Google docs as our method to share and to track progress. Describe the tasks required for each iteration in the 2.1.6.2 Iteration Plan and allow anyone to work on any task.
Uniformity of software applications across computers is lacking leading to compatabiliy issues. Also want version control and ability to share documents.	Utilizing Google docs so that all documents can be accessed by any computer without losing any work. Purchase needed software.
Computer in conference room disappears, and purchase of replacement is not justified.	For now we've moved meetings up to office 301.
Collaborators may not correspond with us.	Create new collaboration with people that are committed to the project.
Use case data may not be quality data	Obtain new use cases that are quality and can be used
How do we get examples of poorly prepared samples to simulate debris, dead cells, poor sample preparation?	Create poorly prepared samples from synthetic data set or get from collaborators and adjust algorithm/metric to flag these samples for manual analysis.
We may not have clear goals regarding the use case dataset.	Refined Experimental descriptions and SOPs for gating. Identify and communicate goals for each use case dataset
GvHD and SIV contributing lab's FlowJo workspaces are from the Mac version (.jo files).	Discussed at length in Engineering Meetings and Project Meetings. Status defined by Adam in October is that Java version of FlowJo will not sufficiently read in the .jo files and a utility is required to automate the loading of exported populations from Mac FlowJo workspaces and populate the FlowDx repository and database.
GvHD Use Case definition is only one of 121 populations. The contributor's FlowJo Mac workspaces have to be trimmed of extraneous FCS files and populations.	This will be handled manually or we can request that the data be re-analyzed by the submitting lab.
SIV FCS files are named identically for Timepoints 2-6 causing issues in FlowJo referencing wrong FCS files if workspaces are moved between folders/ computers. De-identify MonkeyIds	Repository and Database will give each FCS files a unique identifier and using remote FCS files from the repository with "seed" workspaces will resolve the issue.
Trying to use FlowJo beta builds to learn FlowJo, cytometry and do analysis. Delays in analysis due to buggy builds.	Stay with more stable version and have a FlowDx branch in Eclipse for FlowDx project use.
Once we design and implement a FlowDx solution for a researcher, what will it take to upgrade to the next version of FlowJo?	Software validation through quality assurance testing for version upgrade.