So, we just talked about six different types of clinical decision support tools that you could be using. How do you decide which tool to use? Which technique is the best to solve the problem at hand? I think this is really a hard challenge to do trying to understand what is the right fit for your problem, how complex is decision being made, is it's something simple that it may be a calculator can provide? Is it a straight forward rule that you can do with a conditional logic or does it require more complex decision-making in understanding in education where you might have to have a whole user interface and provide people with workflow and different types of visualizations to help them understand that? So, I think this is really a hard challenge to address trying to find what's the appropriate solution to this problem. What I want to introduced is a case example of how we went about trying to solve a complex problem and how we went through the different phases of implementing that solution. So, the example I'd like to provide is one around the decision support tool that we constructed for prostate cancer. Prostate cancer affects over 180,000 men are diagnosed with it every year. One in nine men in their lifetime will be diagnosed with prostate cancer. It's certainly very scary to be able to get a diagnosis of cancer and trying to understand all the decisions you have to make around whether you should have surgery, whether you should have chemotherapy, or radiation therapy and trying to be able to effectively control that disease. However, about 30 percent of these cancers are indolent tumors that do not require surgery. They're growing very slowly or they will not impact the patient's outcomes. There are side effects for having surgery which might affect urine function, bowel function, sexual function. So, of those, we still see majority of men still opt for surgical removal or radiation therapy at a cost of over $30,000 procedures. So, we know that these patients don't necessary need surgery and they're having surgery. Anyway, this has a total cost of over $1.2 billion a year at 40,000 unnecessary surgeries. So, the questions are, why are men who have indolent tumors or tumors are growing very slowly are still opting to have a surgical remover or radiation as opposed to active surveillance? So, we have a very strong research group in prostate active surveillance led by a urologist Dr. Bal Carter and medical oncologists Kenneth Pienta who since 1995 started tracking patients with low grade carcinomas in their prostate. They cut over 1,600 patients who were able to track them over time. Many of them tried to say inactive surveillance, they didn't necessarily feel they wanted to have surgery. This was tracked through PSA label which is a prostate-specific antigen blood test as long with MRIs of the prostate which is an imaging technique that can look at suspicious tumors as well as biopsies on those tumors that can be looked at by a pathologist to actually classify them into a grade group. With that, they partnered with the school public health and Dr. Scott Zeger and Yates Coley and built a Bayesian model to build an estimate to, build a probability of what's the underlying risk of disease, and the estimate of treatment effects. Should I have a biopsy? If I have a biopsy today, what is the probability that my tumor will be at a higher grade group or more aggressive than it is currently and what would be my prognosis over the next five to 10 years? So, they built this Bayesian model and they really helped in trying to think through those problems. So, to address this problem, I'd like to talk about the design thinking process. The design thinking process is a human-centered approach to trying to understand a problem. It's really trying to understand how humans work together and really how they tried to approach solving different tasks and look it using technology. So, at the heart of design thinking is empathy and trying to understand the problem. So, we start with designers working directly with our clinical teams and they often like to go in and observe the workflow. So, they actually would sit in with patients and physicians with their consent to understand the discussions that they're having to be able to formulate questions that they can then create interviews with the physicians and the patients. So, we actually had a physician user group and a patient user group. We would do one-on-one interviews and focus groups and surveys to ask them what questions are they asking? What sort of features where they'd like to have in this tool, and what decisions are they trying to make? I think the key rule of them here is really to understand how people behave and not how you wish them to behave. So, this really is process of trying to understand that healthcare is very complex systems and trying to understand how people are interacting with it also can vary. I like this model of a work system and how interactive and dynamically interactive it is. It involves people coming together to perform tasks. This can be the patient and the physician as well as nurses or technologists and they perform tasks together, this could be a consult with the patient to review their urology history, the use of tools and technology like medical record, but also it's under the constraints of an environment and an organization. I think, it's really important to be able to consider all these constraints. When you're designing a clinical decision support tool to see that it makes sure it's the right fit for your environment. This can include really understanding whether it needs to be on mobile phones or mobile technology for care providers that are highly mobile. It could be workstation based like a radiologist in a reading room, it could be in a clinical workstation, in a clinic, and it could be also device oriented in different types of settings. There are really a large array of different professionals working in the healthcare system from technologists, nurses, and physicians. There are different specialties of physicians working within a health system as well as at different levels in an academic medical center. We also have trainees as medical students, as residents in training, as well as attendings and fellows. I think it's also important to consider really for the user protective of their technical proficiency with using technologies. Some are really technophile and they really like to have a lot of features on their solutions whereas people who are really not interested in being a technophile are really just looking at the usability and want to have a minimum features as possible so it makes it easier to learn the tool as opposed to having all these advanced features to use it. I think another key feature there is the frequency of use. For someone who's using something say 10 times a day and needs to have a different look and feel than someone who's only using this tool once every 10 days. So, it turns out learnability. If you're having to use a tool very infrequently, learnability is very important in being able to work with it. Being able to define all the different tasks that you're trying to ask them to do whether they're ordering tests or prescribing medications, performing procedures, reviewing results, all these are in that context and understand the environment you're in really defines a lot of the technology you can use whether it's a sterile operating room where you have to use cleanable serializable surfaces to remote telemedicine setting where you're going to maybe dependent upon a patient's mobile phone or their internet provider. So, understanding the technology constraints around the different settings that you're in is very important as well as some of the organizational environments. In an academic medical center will have a different environment and different access to different advanced imaging techniques. So, our procedures and scanners then perhaps a rural private practice environments. So, being able to understand the setting you're in also can affect the decisions you make for building a decision support tool. So, for our prostate active surveillance tool, we're looking at a clinical workstation. We know that the providers are placing orders and looking at the patient's medical record using electronic medical records, so we want to have something that's integrated well into their EMR where our users, our urologist and a patient. We are looking at the task of reviewing results together and evaluating options along with the environment as a clinic outpatient setting and as part of an academic medical center for this project.