Cancer Patient Lab Expert Webinar

Disease Modeling: Improving Cancer Diagnosis and Treatment Outcomes

Featuring: Michael Liebman, PhD

In short

Dr. Michael Liebman, a disease modeler with experience in genomics and cancer diagnostics, explains why standard diagnostic and treatment processes often fall short — and how building more detailed, individualized models of disease could lead to better outcomes. He uses real examples, including a heart failure trial where one drug was tested on what turned out to be five distinct diseases, to show that oversimplified approaches can hide the complexity that matters most for each patient.

  • •Two patients with identical test results can have very different diseases and respond very differently to treatment — it is reasonable to ask your doctor what is known about your specific disease subtype, not just your diagnosis category.
  • •Biomarkers measured at a single clinic visit give an incomplete picture; ask whether tracking changes in your results over time could add useful information to your care.
  • •Doctors work under real time and economic pressures that can shape their recommendations; it is fair to ask how confident they are in a diagnosis or treatment choice and whether other options exist.
  • •If a treatment stops working, the underlying reason may be that your disease is more specific than the original diagnosis captured — this is worth raising with your care team as a question, not a challenge.

Ask anything about this — free, no signup

Instant answers grounded in real guidelines, not the internet.

Richard Anders and Brad Power September 7, 2022 “Biology is about systems and processes… but the process of interacting with a patient, with a physician, is actually more complex (than the central dogma).” Michael Liebman

Meeting Summary

Patients, caregivers, and physicians trying to manage a difficult medical journey are desperately seeking resolution, but often they encounter frustration and failure instead. While some patients follow the prescribed route and have great outcomes, others progress down the first-line, second-line and beyond treatment alleyways, yet what they see in front of them always seems to be another alley, or worse, a wall. Dr.

Michael Liebman’s thesis is that we are led down these frustrating alleys because we have overly simple conceptions of the process we are in, and do not understand the nuances as they affect us.

•Disease Treatment : A patient has symptoms of disease, gets a diagnosis, a treatment, and has an outcome. But this process is actually much more complex. It depends on who the patient is, how engaged they are in their treatment, what they bring to the physician, the environment where the physician works, and the country and culture which surround them. It also depends on who is looking at the process, i.e., the patient, caregiver, physician, pharma, regulator, payer, or researcher. Each stakeholder has their biases. For example, physicians have to be very operational, especially based on economics and time pressures. They are also considering what needs to be done for reimbursement, which may not follow guidelines. Patients want an answer before they leave the doctor’s office. And the treatment process depends on when the clock starts. Usually the clock starts when the patient shows up in the doctor’s office with symptoms, but there is active research for earlier markers that aren't being used.

•Disease Biomarkers : While each of us generates a practically infinite set of measurements continually, what is actually measured is an eentsy subset of that – only the accepted things which can be easily and affordably measured – and even these are only done episodically on say, the next visit to the clinic. It’s like trying to explain a day- long wild melee with a couple of grainy pictures. While they capture something, much is lost. Two patients may present with the same biomarkers or clinical variables, but progress very differently over time. And two patients that are progressing exactly the same may not present with the same value because they were observed in the doctor's office at different points in time. Patients with similar symptoms may have very different illnesses.

•Drug Development : The target for drug development should be a mechanism, something that enhances our understanding of an individual’s path through a disease, but we're frequently dealing with phenotypes (attributes of the patient). For example, a clinical trial showed no effect over six years with a very large population, and analysis of subgroups did not find any groups for which this drug was appropriate. The problem was the definition of the phenotype, and then using that phenotype to enroll patients into that trial. Subsequent analysis identified five completely different diseases. Dr. Liebman challenges us to consider patient testing and treatment decisions as part of an enormously large and complex multidimensional system. This perspective will lead us to consider ways in which the conventional learnings we get from much simpler analyses (like the gold-standard randomized clinical trials, with their own serious biases) can lead us astray. Instead he asks us to build more robust models of disease, using techniques well-known to the modeling profession (e.g., mathematics, Bayesian and other statistical techniques). Then, looking creatively back at existing data, and creatively forward to generate ideas for new measurements, see how these models can be applied to give more granular views of each person’s condition, and therefore more specific ideas for treatment. This is a grand vision. In his talk, which is included below, he discusses the problems and opportunities of models of systems and processes, giving concrete examples of their application and many ways in which the conventional wisdom relies on some dubious foundations. In this, he is encouraging patients to not be afraid to (respectfully) challenge these learnings and perhaps find a better path for their own care. Key Principles to Avoid the Frustrations of a Simplistic Process Model

•Expand the Scope : Think in terms of systems and processes that are bigger than your main focus and link them to everything else. Include temporal relationships that are commonly ignored. For example, how early should we identify a person as entering a given disease path, and what systems, including more peripheral effects, might be relevant? Consider integrated health providers.

•Test the Model: Enable clinicians to evaluate models retrospectively against their patient populations so they can test performance and see if their patient population may exhibit unique characteristics that through feedback could further enhance the model. Individual physician practices may encounter unique patient populations or more diverse populations and for them to gain confidence that a new algorithm might be appropriate, they should evaluate it against previous patients. If it is not effective, then feedback to the developer can aid in refining it, and/or identify unique characteristics of that physician's patient population.

•Humility: The physician likely does not have all the answers and should share their degree of confidence in any specific decision or recommendation in a spirit of collaboration. Transparency and willingness to accept some degree of uncertainty can help form better patient-physician relationships/interactions and lead to better and more informed decision-making. Physicians are focused on finding the right drug for a patient with a diagnosis. Some physicians will admit that they don't trust a diagnosis, but they don't have the option to try to change that. This is an opportunity for empowering the patient and for research. The information and opinions expressed on this website or platform, or during discussions and presentations (both verbal and written) are not intended as health care recommendations or medical advice by Cancer Patient Lab/Prostate Cancer Lab, its principals, presenters, participants, or representatives for any medical treatment, product, or course of action. You should always consult a doctor about your specific situation before pursuing any health care program, treatment, product or other course of action that might affect your health. Meeting Notes Prostate Cancer Lab Meeting - For Precision Medicine, First We Need Accurate Medicine (Michael Liebman) Wednesday, 9/7/2022

•58:05 SUMMARY KEYWORDS patients, disease, physicians, questions, drug, observations, condition, clinicians, problem, understand, diagnostics, groups, guidelines, treatment, diagnosis, process, phenotype, early, complexity, test SPEAKERS Michael Liebman, Jeff Waldron, Brad Power, Brian McCloskey Michael Liebman 00:20 I'm a modeler. I'm trained as a theoretical chemist. I'm not a clinician. But I have experience, both in industry and pharma as Global Head of Genomics and Computational Biology for Roche. I worked on the original HER2 testing with Abbott Diagnostics’ Vysis Group. And I ran a breast center for the Department of Defense, where we did everything from surgery to molecular studies, and I have some academic appointments as well. We've all heard of the central dogma for molecular biology, which is that DNA goes to RNA and then goes to protein. There's also a central dogma for medicine, which is that first a patient has signs and symptoms Modeling Disease” We've all heard of the central dogma for molecular biology, which is that DNA goes to RNA and then goes to protein. There's also a central dogma for medicine, which is that first a patient has signs and symptoms of disease, and as you all know, they then progress to get a diagnosis, a treatment and have an outcome. But this process of interacting with a patient, with a physician, is actually more complex. Because it depends on what the patient is bringing to the physician. And it also depends on the environment where the physician works. As a point of reference, I'm co-leading an EU effort right now in cardiovascular disease, where we have 30 partners across 15 countries. And you can imagine the complexity of this, across all of those different countries and research groups in just a single disease. The takeaway that I want to leave you with is remembering that biology is about systems and about processes. And it's always much more accurate to describe change than to describe something in an absolute state. That's just a general scientific observation. You may be familiar with the term “phenotype,” and you may have encountered it in your personal healthcare journey. Phenotype refers to an individual's observable traits, such as height, eye color, and blood type. We're going to introduce the concept of a “next generation phenotype,” which refers to how an individual progresses from a non-disease condition into a disease condition over time. That will enable us to bring higher resolution into disease subtyping. Michael Liebman 03:58 An endotype is the subtype of a healthcare condition which is specifically defined by a mechanism. As has been written recently by [FDA Commissioner Robert] Califf, and others, a mechanism really should be the target for drug development. But one of the challenges we have is that we're frequently dealing with phenotypes, not something that enhances the resolution, and understanding, of an individual’s path through the disease. And so that's what I want to try to address. I'll give you an example of what I mean by that. Here is a clinical trial. The outcome showed no effect over six years with a very large population study. Statistical subgroup analysis to try to recover subgroups did not enable them to determine any groups for which this drug was actually appropriate. The problem was the definition of the phenotype, and then using that phenotype to enroll patients into that trial. Because when we applied some novel methods, we found what you can see here: There are actually five different progressions, meaning that there were five different populations, and they never should have been treated the same way. And not understanding that that complexity existed in the population meant that the drug may have worked for some of these subpopulations, and not others. But it also indicates that – in this case it was a heart failure drug – these patients never should have been treated the same way because they don't fundamentally have the same disease. And that should back up not only into the clinical trials, but into drug development. You need to understand that the mechanisms for these five diseases, or five presentations or subtypes of diseases, are actually very different. I'm a modeler with a very coarse model of what a patient journey looks like, and your experiences are a lot more detailed than this. But essentially, you're going from a risk assessment to a diagnosis because of the presentation of symptoms, or clinical signs, with the physicians actually measuring into that diagnosis, treatment, and then a variety of different outcomes. But in reality, you're just cycling back to become a patient again, into possibly another disease or another state. The thing that you need to understand is that, in all of this, You need to understand the question that needs to be addressed. As an economist observed: “If you don't know how to ask the right question, you're not likely to discover anything that's going to be of significant value.” And the question in the patient journey is going to be very different, depending on who's looking at that patient journey. Because each of these groups is emphasizing a different part of this journey, based on what they do, what their daily activities are, and what their priorities are. And none of them are looking at that entire patient journey. What we've focused on is building a model of what this patient journey looks like, the complexity of that, and then allowing different groups to have their individual views, but also being able to expand upon their views, to see how other factors may be actually interrelated to what they need to know. Michael Liebman 08:30 We initially use an ontology, which means we're looking at those concepts, and figuring out all of the factors that are critical for those concepts. This goes many, many layers deeper. You can see what we built is general, and because we can use it across any disease, is something that's disease agnostic. There's no element in here that's specific to any given disease. This can become a model for how we integrate the data to represent and be able to analyze the disease. It also gives you an idea of the kinds of resolution we're looking at. We frequently talk about comorbidities, but comorbidities could be active comorbidities (in other words, another condition that you have, even as you’re being treated for another one, in this case, prostate cancer), or they could be inactive (i.e., previous comorbidities that may have been managed), and they may even be undiagnosed. The complexity of this patient journey is something we try to encapsulate in this form, and we convert to a knowledge graph. There's an ambiguity in diagnosis and staging. If we take a clinical variable, and we take three patients, and we observe them when they come into the clinic (this is for simplicity, it scales up to many many variables), we see two patients may present with the same biomarkers, or clinical variables, but progress very differently over time. And two patients that are progressing exactly the same, may not present with the same value, because they came to be observed in the doctor's office at different points in time. This is a complexity that physicians are always dealing with. Margaret Hamburg from the FDA used to say that clinicians have long observed the patients with similar symptoms may have very different illnesses. It's something that's known, but it impacts a lot of what we need to do to understand the disease. We need to understand that disease is a process that evolves over time. It's not a state, although we tend to treat it as a state in terms of a diagnosis, but a disease process evolves in very, very high dimensional space. Any of your clinical variables, or even your non-clinical variables (environment, lifestyle, social determinants) can impact you as well. What the physician would really like to know is, what's the direction of this? Let’s call it a “vector”. How far are you along that vector and how quickly are you progressing? That is the ideal set of questions a physician would like to ask. But they don't have a way to actually get those details. Mathematically, that means this isn't a vector at all, it’s a tensor [an algebraic object that describes a multilinear relationship between sets of algebraic objects related to a vector space], but we don't have the data to support that. Yet we know that if a patient comes in here, just as I showed in that previous slide, they may get a different diagnosis than if they came in at different numbers along the “disease process” vector, and be managed differently, and have different outcomes. One of the challenges that we have is that we're always looking for biomarkers, clinical variables that we can use. Ideally, if we want to answer these three questions, we need to start off by knowing what this vector looks like. If we don't know what this vector looks like, then we don't know where a biomarker may be changing. The value may be at different points on this vector. That also means that a lot of the biomarkers and diagnostics we have are available because of the technology that allows them to be measured. As I noted before, we can see changes in them. But we don't necessarily know where they are appropriately aligned with this disease process. Michael Liebman 13:28 That's a challenge that we tend to bury in the complexity of the problem, because we don't have a good way to try to tackle it. We don't have the necessary data necessary to do that. Those are some of the things that we're focused on and trying to address. How do we optimize treatment? We don’t have what we call a closed solution. But it's an iterative process to try to continually refine based on the gaps, the limited information we have, and how to get a more accurate profile of the disease for the individual. Phenotypes, as I identified before, have the following limitation: if we have a set of observations, and we have two patients for whom those observations are made, they could be symptoms. clinical observations, or laboratory studies. We know that they will exhibit somewhat different values. And over time, they'll end up in somewhat different places. What we do right now is take a specific diagnosis or a specific response to a treatment or a specific outcome and define that as a specific constraint and then look to see where patients who have that characteristic may have common observations.The idea would be, if we have patients say, with prostate cancer of a certain type, what are the observations that different patients have with that diagnosis? But the issue is, these observations only occur at certain points in time in that progression, not throughout it. The overlap is something that we're correlating with the outcome that we're superimposing on this population. What we've said is, the reality is this: Those observations (on the left side of the slide) are correct. But they're not complete in describing how those patients are progressing. Because while those patients in that phenotype all express that set of attributes, they have different attributes in the time period before and after. How these attributes have changed are different among these patients. If we refer to these patients as having that same condition, we will obscure the fact that these patients are progressing differently. This is what we use to generate what we call a next generation phenotype. As I showed you in that earlier slide, which was for heart failure, where we identified five different patient progressions in the population that was being studied in that clinical trial. We developed algorithms that do this, and some of them are now running on a quantum computer. We use it to look at not only clinical trials, but diagnostic guidelines, and eventually drug discovery as well.The other factor we need to consider is that we are not making continuous observations yet. The promise of some of the digital technology is to give us continuous observation. But that's a long way off. We now make observations at some point in time, and then talk about progression over that time course. But that progression is basically averaged. What we're looking at, in most cases, is from the physician’s perspective, taking a normal distribution of that observation. Michael Liebman 17:58 At any point in time, does that patient exhibit outliers from that [average] distribution? This is frequently the focus of what most physicians are looking for when they're running a chem-20 (a regularly ordered group of blood tests that gives important information about the current status of your health including glucose or blood sugar levels, kidney function, cholesterol, sodium and potassium levels' and more), or other kind of panel. You may have outliers, but that normal range is going to be potentially biased by whatever population was used to generate those initial measurements. And the reality is this: Here are two patients represented as never being abnormal, always within normal range. But you can see by the discrete measurements within the normal graph that their clinical values are moving in different directions (i.e., Patient 1 has decreasing clinical values while Patient 2 has increasing clinical values). We've shown that using these kinds of tests, and taking into consideration the discrete values, rather than looking for the outliers, allows us to discriminate as early as possible when these patients may have been on this path, rather than on that path. That's important for knowing how early a diagnosis could have been made, and how early a patient could have been assigned to the right trajectory, a trajectory that might better align with the treatment needs than the “average” treatment that they're getting right now. Here we are talking about the disease process as the main problem. Every patient has on the order of up to five comorbid conditions, and frequently up to about 10 other drugs or supplements that they may be taking that are independent of what's being used to treat this condition. What that does is alter the trajectory of the disease path, and the impact of all the diagnostics and the response to treatment as well. To give you an example, from a study we did in Italy, we did an analysis of patients who have diabetes and hypertension. And for those of you who aren't aware, about 75% of the patients with diabetes have hypertension, and about 75% of the patients with hypertension have diabetes. We analyzed the records for all of the drugs and all of the medications that these patients were taking. We found that on average, approximately 67% of the patients who had either hypertension or diabetes were getting medications that were contraindicated for hypertension and diabetes, because they were being given drugs for other specialties and other conditions. That's part of the constraint of how medical practice takes place. It's also a result of how clinical guidelines are being established and being implemented, especially by the specialty groups, because they're looking at focus on individual conditions, not the patient in a more holistic manner. There are attempts to try to understand what progression may happen. This is from a study we're doing with multiple sclerosis. We're going back into a patient's EHR record. These are all the observations in the EHR for these MS patients. You can see the proportion of patients with different types of MS diagnoses, who had some of these pre-condition observations in their record. But this is not adequate, because what you really want to know to define the trajectory of that patient is, when these occurred, in what order they occurred, and which ones occurred.. This is not how we're currently looking at a lot of these conditions. I will now turn briefly to prostate cancer examples. Here we are looking at risk classifications, and what you see is what we call an ambiguity, because you have effectively different stages associated with different risk levels. You can also see that risk assignments in different guidelines are not uniform. [For example, you can see that a T2c patient would be intermediate risk under the NCCN Guidelines, but high risk under the Harvard, AUA, or EAU guidelines.] Michael Liebman 22:57 This requires a different kind of harmonization that is currently being performed but is indicative of the fact that these are guidelines and not absolute definitions of conditions or risk factors. This slide is hard to see, but it shows how these elements are defined. The important takeaway is that there's a great deal of subjectivity in assigning these. But more importantly, there's an ambiguity: you have multiple stages that may have the same definition, but different parameters that make that up. You need to keep two things in mind: First, while the staging may be considered as a primary factor in administering treatment, the reality is that when we've analyzed this in breast cancer, because that's where we've done this in more detail, what we found is that, not surprisingly, patients do not necessarily go through this in a linear manner. In other words, you don't progress from to 2A to 2B to 2C and then to 3A. In fact, we have no idea right now how they progress, because a patient at the time of diagnosis has to be treated as it would be unethical to not treat in order to learn the natural course of a patient who is not treated. And second is the ambiguity in defining these [stages] may actually be obscuring the ability to understand what those discrete pairs are. And so, among the things we've done, is to try to reestablish with these discrete measurements how to redefine some of the stages. And we've applied Bayesian analysis to actually identify some critical paths of progression in breast cancer. But there's no reason to suspect that that's unique to breast cancer, because it's a general set of characteristics. Brad Power 25:36 I'm curious about the modeling process and measurement. What's been your experience or lessons from modeling in general? Michael Liebman 26:22 I'm trained as a theoretical chemist, but I've never been in a chemistry department. I don't think they would let me in a chemistry department to be honest. My interest is in solving problems. The 4-hour workshop that I mentioned was for a society for decision professionals. I'm interested in how decisions are being made, the adequacy of the data and knowledge used to make a decision, and the decision process. A lot of those things are very fundamental. The purpose of going to the ontology, which I told you is disease agnostic, is because I'm interested in looking at root cause problems. Seeing that a problem that someone may think is very, very specific, actually represents a classic problem, that maybe someone in another discipline solved, and doesn't have to be solved again. Can we borrow that technology, adapt that technology, and learn from the experiences in those other domains? As a short example, we used a model to simulate biological pathways. But usually as a chemist, we would model a pathway using kinetics, and set up differential equations, and rate constants to simulate the behavior. But we don't know any of those parameters, they're not measured under the same condition. So what we did here is consider that a biological pathway is very much like a communication network, a series of towers that you have moving signals around. Biology is just moving a signal around to accommodate blockages in vision, mutations, and so on. We went to the engineers and borrowed their communication engineer tools to allow us to simulate that behavior, because it's exactly the same problem, just implemented in a different way. We could ask questions in a qualitative way that we really needed to answer while everyone else was waiting for quantitative data that we may never get. In modeling, that's part of the approach that we take. Brad Power 29:02 It sounds very similar to the discussion we had on proteomics with Karin Rodland, looking at pathways between the cell and nucleus, such as the MEK pathway, and looking for biomarkers. It also sounds similar to the mechanistic approach that the folks from Clemson were doing. Jeff Waldron 29:40 Diagnostics have been transforming healthcare. But it's getting very hard because there are so many diagnostic options, including next gen sequencing, proteomics, and multi-omics, but also you have all these companion and complementary diagnostics tied to a particular therapy. It's not like a patient presents with a condition or series of conditions. You don’t say, “Oh, that means I need to do this test.” Picking the right diagnostics, including liquid biopsies, is really difficult now. That’s why I see us moving to algorithm-based medicine. The other big trend I'm seeing in the groups I work with is this whole combination therapy versus monotherapy. You're now picking combinations of therapy, which is not as simple as the linear progression of first then second line of treatment, but two or three different drugs at different doses as the first line, or new adjuvant therapy. The algorithms are getting insanely complicated. Michael Liebman 31:14 We're interested in understanding a given drug. For example, in breast cancer, there's HER2/neu, which you may be familiar with as a biomarker, which about 20% of patients overexpress. Part of the problem we identified is that there are two different ways to measure HER2/neu. The reality is, in practice, HER2/neu is supposed to be a companion diagnostic for treatment. But 66% of the women who are eligible for testing are never tested. The practice isn't compliant at all. That's because, as I showed in an earlier slide, it is partially the effect of patient pressure that the drug is available to them because it's less toxic than other drugs. They ask for it. In this case, if you use it, and only 40% of the women respond to the drug, you have the cost of the drug, you have side effects. But the drug is a biologic, and it alters the course of the disease. [Even perhaps in non-eligible patients since HER2/neu is widely expressed in cells in the body.] By the time you find out it didn't work, you’re not quite sure what the next drug is that you need to use. This is a general problem. The other problem the FDA caused is they’ve approved two different tests, an immunohistochemistry test and a genomic FISH test to look at the number of gene copies. One test measures DNA, and one test measures protein. We’ve learned that there are a lot of steps in between, and it's not a one-to-one transition. The problem you run into is what we found when we measured our patients using both tests: the concordance is about 20%. The results are really good if you have a very benign disease, and really good if you have a very aggressive disease, but really bad for the majority of patients in the middle. The reason it's bad is you're measuring two different processes. You may be learning that neither of those is the correct process for the biomarker, but only the ones that are measurable. Brian McCloskey 35:27 You've captured so many of the challenges that patients have, and the complexity of what we're dealing with. There is a slide on the treatment journey that we refer to as the “conveyor belt of death.” We're trying to get off of that. Your discussion today is very, very complementary to the discussion we had with Bob Gatenby, who talked about adaptive therapy and the notion of having a single extinction event for complex disease. You really have to think about it strategically: what process is going on, what will your treatment look like, what's your next treatment, and what are subsequent treatments going to look like? And how is that [evolutionary] pressure going to change the direction of the disease? How are you working with clinicians to actually integrate this approach to problem solving in the clinic for patients? If you look at the dogma, from DNA to RNA to proteins, what's driving those measured changes in proteins? They're just not magically happening. There's something biologically happening. You have to look beyond just those three elements. Michael Liebman 37:03 When you look from a modeling perspective, it could be a change in the protein itself, it could be a change in the level of the protein, it could be a change of when and where the proteins are produced. These are all different factors that go into the function of the protein, not just the fact that it's a protein coming out of RNA. For example, in asthma we have alpha-1 antitrypsin (an inherited disorder that may cause lung disease and liver disease). The lung is being attacked because the inhibitor of the elastase (an enzyme made by special tissue in the pancreas) doesn't show up at the right time. But it's there. And that may be a different condition from when it doesn't show up at all and needs to be managed differently. But we have to start to ask those questions. I work primarily with clinicians. When I was running the Breast Center, I worked with the Chief of Surgery at Walter Reed, and this was during the Gulf War, so as you can imagine he was busy. Getting his attention to do research was not necessarily easy. But what was clear was helping them do their job better, and helping them identify the questions they don't ask is critical. Physicians have to be very operational, especially based on economics, time and everything else, such as patients wanting an answer when they leave the office, not having someone go off to research it. So they have to be very operational and not strategic. What we typically do in most of the diseases is try to figure out strategically what the questions are that they would like to have answered. For example, I'm co-leading a big EU program, with study centers in 15 countries, in cardiovascular disease. It's all with cardiologists, and they start with when the patient shows up. The issue is: how early could you have determined cardiovascular problems before that patient showed up? What is the measurement of risk? They know the way we are doing it now is inadequate. Are they really detecting early disease? Are they protecting risk or disease? And how early could you have determined which path the patient is on so they might have been able to intercede? Some of those interventions might improve patient empowerment, and others might be clinical interventions. Those are the questions physicians want to know in general, but they don't have the tools to ask or try to address them. For example, when we look at MS, the clinicians we're working with are almost solely focused on the issue of “the right drug for the patient with this diagnosis.” They will admit that they don't trust the diagnosis. But they don't have the option of knowing what they can try to do to change that. That's where the opportunity is for research. Also empowering the patient to understand some of that. And just realize that’s the reality, because it is the reality. In MS it's easy to do a survey of patients to ask questions about what they experienced before they came into the physician's office. We're collecting unscripted patient histories. What was everything that you ever experienced that you may never have gone to a doctor for? We want to understand what may have been early signs. It's changing the paradigm, but changing the paradigm is very hard, as you can imagine. Brian McCloskey 41:44 My disease didn’t have any early concrete observations that would have predicted it. Can you go back and model the drivers of the disease manifestation? Michael Liebman 42:15 When you say there was an absence of early observations, you're taking into consideration things that you may report to a physician or tests they may run. The reality is they may not be the right tests. Part of what we're trying to do, for example in the cardiovascular project, is taking all of the existing databases that are out there. They all start with the patient coming in to be diagnosed and treated. We're putting at the front end all of the databases that come from the Netherlands, Norway, Sweden, and Finland, where the Scandinavian countries in particular have been following patients without a diagnosis for 30 or 40 years, to see how they progress over time, without knowing what to expect. We're putting that at the front end to say, “These patients are progressing similarly.” We don't know, maybe they won't get heart failure. But they're likely to present with the same condition at some point. Some of them will have heart failure, so that we'll be able to keep going back earlier and earlier. What are the markers that we might want to now include in a conventional study that we're not including? That's how we're setting this up to be able to drive new studies, by identifying some of these earlier markers that aren't being used, because it's easy to look at the markers we have and know that they are not adequate. Brian McCloskey 44:06 But each one of those databases have certain biases, right? So how do you account for that? Michael Liebman 44:21 We incorporate that. In fact, that's one of the things we're focused on. We're using a unique federated model. And in that federated model, we're actually looking at how each data field has been defined. For example, one of the measurements is glomerular filtration rate (GFR), which looks at kidney processing. All the databases have GFR reported, but there are 5 different algorithms that compute GFR, and they're not equivalent. For example, some include race, and some don't. You need to know what algorithm was used to develop that value to be able to do a meta analysis and incorporate that data, understanding what biases may be in it. If you don't, you're mixing oranges and apples. Brian McCloskey 45:27 There could be a bias in the source. You mentioned you're getting a database from Holland. How would the lifestyle of a Dutch person compare to somebody in Philadelphia? Michael Liebman We usually model things within the database. We evaluate the models across the databases, which gives us an idea. You mentioned Philadelphia. We have a study going on where we're comparing 12 different immigrant populations in Philadelphia, and we are trying to figure out what effects are generalizable and what are unique to those culture groups. Brian McCloskey You mentioned the NCCN Guidelines. Are you working with people at the NCCN Guidelines to elevate what they provide to clinicians and to patients? Because it is very operationally-oriented. It's a series of "if- thens". Are you connected to them? Michael Liebman 47:02 We look at more than the NCCN Guidelines. For example, we look internationally and compare claims data to guidelines. Claims data is what's being done for reimbursement purposes, and guidelines are what are being recommended for practice. There's a very common difference between them. Physicians are responding to what needs to be done for reimbursement, and based on their experience, may not follow guidelines. Having come out of pharma, the highest form of evidence for an evidence-based guideline is a randomized clinical trial. But as I showed you in that very early slide about heart failure, that was biased, because they used inclusion exclusion criteria that didn't approximate the real world population. We're looking from that perspective, we're not working directly with the groups that actually administer it, other than to show where there may be weaknesses. We're trying to enhance transparency. We don't make judgments. We're working to enhance the transparency of certain data that may be less reliable than other data that's being included. Jeff Waldron 48:47 I've been working in expanded access programs (EAPs), which you are probably quite familiar with. My problem with clinical trials is they pretty much cherry pick patients with early stage disease and few comorbidities because they want to get the drug approved. You may be on the placebo arm of the trial also, whereas patients that have exhausted the standard-of-care options and are on their third or fourth line of treatment, if they can gain access through an EAP program to an investigational drug, it may be transformative. I'm trying to work for several companies now to facilitate this because amazingly it’s a basic but immature process where you apply to the FDA and the pharma company, you have to show that you have no other options, and that your physician believes you may respond to this particular therapy. Michael Liebman 49:46 Our approach is a little different. Because the RCTs are all biased for success, as you might expect, when we do our stratification, we don't include the RCT data. We look at real world patient data. That gives us a profile of what the real world looks like, then we go back into a failed trial and re-diagnose the patients and see where the drug would have worked. Jeff Waldron 50:22 I'm trying to convince pharma now that EAP programs are good for patients. I think they should be gathering the data on those patients, because that's what you just said, those EAP patients are probably more representative of the post-commercial target patient population. Michael Liebman 50:45 The one difference is because they're small populations, it's hard to show statistical significance. Jeff Waldron 50:57 That’s true. But expanded, it could become less narrow. Usually it's from academic medical centers where they're already doing the trial with trial patients. Then for patients, they have no other options. They're offering it or soliciting EAP access. Michael Liebman 51:14 That makes perfect sense. We're trying to see how you can expand upon that by understanding that complexity. Pharma has targeted in that entire process what the population will be. The commercialization group already knows where it's going to go. They've started to prepare the marketing materials and everything else, in anticipation of approval. We're redefining those diagnoses, and identifying where the drug may have failed. We can show for a small biotech, how they can go around Pharma because they can identify a niche group for whom the drug would work. It's too small and too hard for Pharma to re-engineer the whole process. So they won't deal with it. Brad Power 52:21 I've been very impressed by your ability to go to first principles, and not just take the process or the conventional, accepted “fundamental” definitions. And I love the word “ontology.” You're very grounded, and very solid on definitions and what's going on. And you come up with very unique insights. I just think that this is very refreshing to show that we have to reflect on every step in the process and not just take for granted some of the things like the process of an RCT, which is “holy, holy” an irrefutable sacred cow, if you will. Any parting thoughts on that in general, on your approach and your lessons from that perspective on these processes? Michael Liebman 53:38 The biggest takeaway we always have is, “the hardest thing to change is culture”. We can have all the evidence we want. But when someone believes that this is the way you do it, it's very, very hard to get those changes in place. And resistance is organized to support a specific mechanism of development and discovery and development. It’s very, very hard to make change, which is why I left pharma, because we couldn't make those changes internally. The patient has to be a spokesperson for their condition, and has to be open to understanding that physicians are 99% doing the best they can. But they have limited information. I'll leave you with one anecdote from when I was at Mount Sinai. I used to have constant battles with the Dean of Medicine, because I was on the basic science side. And so the cultures don't always mix for a variety of reasons. I went to the first day of class for the new medical students, and he said, “We're going to spend the next few years cramming your heads with so many facts that you won't really be able to learn them all. But you need to understand half of the things we're going to teach you are wrong.” And then he said, “And the real problem is we don't know which half.” And that was probably the most honest thing I've ever heard. Brad Power 55:33 I'm thinking of Brian. You've been focusing on some of the culture of pharma; he's been focusing on the culture of providers. Brian leans into two things: more testing is better, and there should be a treatment strategy that goes beyond the hamster wheel of “try one thing”, and it works for 12 months, and try another thing, and it works for 12 months, and keep trying these one things. He wants to have more of a strategy. And he is frustrated that he can't find an alignment with his treating oncologist. The question then is, “Should he just do what he thinks is right, or should he find another clinician?” What would be your advice in that environment? Michael Liebman 56:31 I'm dealing in Europe, with groups that are called integrated health providers. They work internationally. They have a much broader or more holistic approach to treating the patient. An issue we're always going to be faced with, especially now, is specialization. Specialization provides a very, very deep understanding about an area, but it doesn't have enough breadth to look at what those boundaries need to also consider. I can point you to the integrated health group that I am working with. I don't know what their connections are in the US, I can ask them, I'm sure even if they're not formally linked, they probably are aware of groups that would have that same kind of philosophy. And that's probably closer to what you're looking for. Then there’s the oncologist who goes to ASCO and wants to use the drug that they were talking about in the last lecture.

Want to learn more about your specific case?

Upload your medical records and ask Navis questions tailored to your diagnosis.