Counterfactual learning systems

Separating causation and correlation in AI systems is a challenge because most machine learning systems look for trends, but not 'counterfactual' information, which is more like the way we, humans, and doctors think.

People, like doctors, make decision based on what they know to be true and untrue, and build causal reasoning into a diagnosis. Most #machinelearning systems don't build causality: they are built on associations / correlations. We don't care the sky is probably blue when we get a cold, but we do care your T-cell count is low when you get a cold (causation vs. correlation). Counterfactual data? "Lets get a chest x-ray / ultrasound / CBC ..." i.e., some data that rules out other possibilities to see how symptom relates to disorder (directly or indirectly). But what rules can you build for machine learning? (Un)surprisingly this paper shows it can be simple, (because thats how *our* brains probably work): disease should be consistent with diagnosis, rule out stuff that isn't possible, and keep it simple: 1 Dx fitting M symptoms is better than N Dx fitting M symptoms. They go on to define things called "expected disablement" and "expected sufficiency". The former is obvious, but the latter is like "sufficient cause", and state theorems, one of which is that disablement and sufficiency are sufficient conditions for the rules above. But real data is noisy and murks the variables and so there needs to be a way to account for noise. (insert mathy stuff here). Thats all fine, but the litmus test is "How does this compare to actual clinical decisions?" In short, a physician achieves higher accuracy in diagnosing a disorder for simpler problems, and the algorithm outperforms for more complex problems. Thats good for rare disease classification. That makes sense as the story of #machinelearning and #AI in medical diagnoses suggests utility in a role as a 'decision support tool', but not a fully autonomous one. The difference here is that the model behaves more like a clinician would. For you Bayesians ... when you first learned Bayes' Theorem I bet you pondered "Why can't we do counterfactual inference in medical diagnosis? ...policy making? ... court decisions?". This article is a nice progression of how we can use AI based on causation - not just correlation. Don't believe me? Read for yourself.

https://www.nature.com/articles/s41467-020-17419-7?fbclid=IwAR3a6txrtrUtr6IDyaV-Bsc56bbUJmHfGtg3RVoyhMj0LvR-wvuRTm7A7YU

Extracellular vesicle and particle biomarkers and AI

A very interesting article on extracellular vesicle and particle biomarkers and how they might be used in cancer detection.

 

https://www.sciencedirect.com/science/article/pii/S0092867420308746?via%3Dihub

There are gazillion authors from a bajillion institutions on this paper. Collaboration!

The gold standard to confirm cancer and other aliments is a tissue biopsy, where a small sample of tissue is extracted from the suspicious growth. But extracting a tissue sample isn’t possible in many situations, especially when there are other co-morbidities where the biopsy can introduce more problems than it attempts to solve.

So ‘liquid’ biopsies is another approach: stuff like drawing blood, lymphatic/bile, etc., which is not as difficult. But that stuff isn’t where the tumor is… its stuff floating around the body. Some of the gunk that floats around outside the cell are EVPs...or ‘extracellular vesicles and particles’. Basically they’re goops of stuff that float outside the cell, originating from ‘sorters of things’ in your cells. I (probably mistakenly) think of them as recipe pages floating outside the bookstore that sells recipe books. Except there are gazillion (actually billions of EVPs) recipes, and a gazillion books: trying to figure out what page came from what book would seem an impossible task, right? Well… this is where the story gets interesting!

This team used machine learning techniques to sort through all the EVPs based on sizes and other subcategories (mice/human, cancers). They found that the relationship between +10K EVPs and tumors in mice and humans were not the same (interesting since mouse models are used in so much research). They then sifted through all these possible markers to see if they could be used as a cancer detector.

How do you sort through literally 10s of thousands of markers for trends? Reliably? #Machinelearning, of course. They found the presence/absence of 13 common EVPs could be used to classify both lung and pancreatic cancers. But are those little floaters actually associated with tumors? In other words, is there a relationship between biopsy findings and the floaters?

While their dataset was kinda small, they could verify the biopsy findings with the floaters to +90% sensitivity / specificity (sensitivity is how well you can detect something (like how likely you are to stop at a sign that looks like a stop sign), and specificity is how well you can rule all other possibilities out (like how well you ignore the sign that looks like a stop sign but really isn’t). They then attempted to ensure that what they saw wasn’t just stuff you’d seen normally... not a trivial task.

What does it all mean? Maybe *earlier* cancer detection? Increased precision cancer detection? Dunno… but it is super cool that floaters in the blood could be so precise in detecting disease. These EVPs may be echoes of the body saying ‘something ain’t right’. We didn’t have the tools to be able to appreciate this signal until we developed the technology to detect the echoes.

Super cool.

Meet RoboBEER

Meet RoboBEER, a robotic beer pourer.

As you know, the demand for high quality beers worldwide has exploded over the last few decades. What drives quality? Well one way to discern quality is to objectively characterize features within the beer.

What features you may ask? Some of them are visual, like the color and foam-ability, such as maximum volume of foam, total lifetime of foam, foam drainage, size of the bubbles in the foam. But not just any idiot can pour the beer, as a Guinness lover will tell you, since a good pour is crucial. Fortunately RoboBEER can pull the ‘perfect’ pint: RoboBEER pulls 80 mL (+/- 10 mL) while monitoring the liquid temperature, assessing the alcohol and CO2 levels, all through your kids Arduino control board and a Matlab interface (yeay Matlab!).  

But what about more important features like taste? Surely no robot could do that right? No way. But… maybe you could predict things like mouthfeel from all the features obtained in by RoboBEER? You could capture descriptions of taste from experts through a questionnaire: 10 basic categories: bitter, sweet, sour, aroma in grains, aroma in hops, aroma in yeast, viscosity, astringency, carbonation mouthfeel, and flavor hops. Then, have them sample twenty-two beers. (What I would do to be a part of this study!)

Could you train a neural network to predict what the beer would taste like just based off the data from RoboBEER?

A ‘feedforward’ neural network was designed where, essentially, you take all the inputs from the RoboBEER (head size, color, etc), and the outputs from the tasters (bitterness, sweetness, mouthfeel) and see if a neural network can predict the taste based on those inputs. You do some fun math like principal component analysis to help with sorting all the data and patterns, pump them into the network for AI training and what do you get?

For the independent testing data, the AI system from RoboBEER data could predict what it a beer would taste like with an accuracy of 86%. What does this mean? Well… very likely, RoboBEER is a better judge of beer than you are. And it doesn’t even have to taste the beer.

Don’t believe me? Read for yourself.

https://onlinelibrary.wiley.com/doi/epdf/10.1111/1750-3841.14114

What is your best workplace productivity hack?

I got the bright (read crazy) idea that it would be a lot of fun to organize a workshop where people can share tips and tricks for improving your workplace productivity. So when the call came out for workshops for the COMP ASM in Kelowna, I thought why not have a workshop dedicating to improving productivity?

I am a big fan of the "GTD" workflow. If you haven't read it and want to learn some tips on improving workplace productivity, you really should pick up the book. But once you have, you then really ought to check out Luc Beaulieus' blog, particularly his digital GTD series. So after connecting with Luc and Thor Bjarnason (and the antithesis of Luc's love for Macs!) we cobbled up a proposal for a workshop.

Productivity implies the ability to yield a high degree of 'output' without expending too much energy. This assumes you have quality inputs and an efficient process which yields, ideally, high quality outputs. The metrics of outputs are important (e.g., number of publications, number of quality assurance checks performed, % increase in salary, etc.,) but also important are the inputs. The old saying "Garbage In = Garbage Out" applies... but generally, when it comes to Medical Physics tasks, we are good at conditioning the inputs (i.e., taking good measurements, or filtering important queries from colleagues, etc.). And often we have good strategies in processing those inputs and yielding good outputs. But can we do that processing more efficiently (i.e., improve your 'workflow')? One of the goals of this workshop is to shed some light on some optimal workflow strategies for different types of inputs.

But digging deeper into the topic got me on to something: One of the things we may not appreciate when we think "productivity" is the influence inputs have on things outside of the process itself. What I mean by this is that even the basic human needs, such as shelter, sustenance, the need for belonging, and friendships can have a huge impact on your professional productivity. Furthermore, while you may have great strategies for dealing with a barrage of e-mails and QA tasks, those 'inputs' could affect your well-being. Strategies like closing your door for an hour a day, or (one of my favorite techniques) having a 'personal time-out', can not only help you process things efficiently, they can improve overall well being.

I'm really looking forward to sharing some of our strategies for improving workplace productivity with our Medical Physics community, and, more importantly, learning from participants of their challenges and strategies in managing life as a Clinical, Research, and/or Administrative Medical Physicist.

Here is the link to the workshop details.

After the workshop, maybe Luc, Thor and I will cobble together an article for the COMP Newsletter or something.

See you in Kelowna!

My latest paper on Computing Science

After many hard years of work, I believe I have published my Magnus Opus. After 20+ years of being a scientist, this is the most challenging and important work I have ever done. Through the incredible efforts of myself and several of my colleagues, I humbly offer this work.I would be honored if you would read this.

We  have elected to publish this work anonymously, under the pseudonyms of DUNE characterless.

The paper is accessible here:
https://drive.google.com/open?id=1O-VTvM0i4ZzAgWxAtQPAmuky5ktOLvkp



(seriously... https://pdos.csail.mit.edu/archive/scigen/#about )

International Day of Medical Physics Crossword Puzzle - 2017

Once again our department is organizing a few interesting activities for IDMP (see www.iomp.org for details).

Here is a harder crossword puzzle for this year... last year was too easy: this one will test the limits of medical physics knowledge (or perhaps your googlifying skills!).

If you're interested, I used this online tool to create the crossword which can neatly create a word or pdf of your puzzle. Enjoy!

My venture into Wikipedia

Backstory

Long ago when I was the Editor of the COMP InterACTIONS Newsletter I wrote an article about (among other things) Wikipedia (see here for the publication). Back in 2007, Wikipedia was still relatively new and there was good reason to be skeptical of the published content. Relevant excerpts from that article are reproduced below (with post notes and inserted links):

"...Out of curiosity, I typed in the words ‘Medical Physics’ in Wikipedia. After following a few links, I came across web pages that had the words ‘EDIT’ alongside each of the written texts. I clicked it and started to feel like a blind, thumbless plumber stumbling towards a CT with the gantry covers off (incompetent). After stopping myself from inflicting damage, I pondered: who are the contributors of these rather finely crafted words? 

After seeing some rather anonymous contributors, I noticed that characters by the names of ‘kungfuadam’ and ‘Bobo the Ninja’ appeared to be editing some of the pages. It turns out that Bobo is a self-proclaimed expert in chemistry who attends Rock Bridge High School in Missouri. I don’t know what kind of journals you are reading, but I haven’t cited or read many works from Bobo yet. [Post note: Bobo is gone! or has changed identities!] 

Some interesting facts about Wikipedia:
• Wikipedia is a free internet encyclopedia that anyone can edit
• There is a entry of ‘Wikipedia’ on Wikipedia
• According to a peer reviewed study published in Nature, Wikipedia is as reliable as an encyclopedia in the accuracy of information provided (something the traveling Britannica salesman failed to mention).

To me, Wikipedia is a shining example of how good -and bad- things can get when you mix science and everyday life. Theoretically, if you wait long enough, any falsehoods, or ‘graffiti’, would be replaced with a majority consensus opinion on the “correct” description. Interesting concept. I just hope those volunteers adding to the website are competent. Sounds like digital Darwinism. Stephen Colbert, a comedian/political satirist calls it “bringing democracy to knowledge”, which I found particularly funny, and scary. [Post note: Stephen Colbert performed a hilarious experiment with Wikipedia]"

Re-reading this didn't really change my opinion all that much, but I think it is fair to say that the growth and reliance on Wikipedia as a reference-tool has grown tremendously since 2007. That isn't to say that Wikipedia have no challenges. For a real mind-bender/meta-exercise, consider that there is a pretty good submission on the criticisms of Wikipedia on Wikipedia. In fact, Time magazine thinks it is in trouble. Surprisingly, they stated the following:

"The problem, most researchers and Wikipedia stewards seem to agree, is that the core community of Wikipedians are too hostile to newcomers, scaring them off with intractable guidelines and a general defensiveness. One detailed study from 2012 found that new editors often find that their first contributions to the site are quickly rejected by more experienced users, which directly correlates with a drop in the likelihood that they will continue to contribute to the site. "

I found this particularly worrisome after realizing that someone needed to create an entry for Jack Cunningham somewhere on the internet.

My venture

If you don't know Jack Cunningham, well... you can now read his profile on Wikipedia. My motivation for creating an entry was based on the few times I searched him through various search engines. Too frequently I'd be disappointed with the results. So I figured why not enter him into Wikipedia? It is citable and a probably a good thing to do. Plus I could add some of my medical physics competency into this massive digital-Darwin experiment.

The steps were quite easy. 

• Create a profile in Wikipedia
• Read what you need to do to create your first article
• Follow all the suggestions in that article
• And write it up in your draft-space!
• Get all your references lined up
• Write and edit.
• Edit some more
• Then submit it for review

I had some experience with HMTL coding so using mark-up language to create the article wasn't much of a challenge. There are a great number of helpful editing tools which can make referencing pretty simple.

For the record, my first submission was rejected due to poor references. Easy-peasy fix. The second revision took much longer to receive feedback, but when it came, I received a lot of great feedback from the community. Shortly thereafter it was published! I received a bunch of helpful suggestions such as inserting links to the profile (which I did through inserting a link on Harold John's wiki entry). 

Now that the page is created, anyone can edit it. And that kind of scares me a bit. Maybe Bobo The Ninja is hammering away at the page at this second (probably not). Now that I am an editor of a page, I can receive a message / flag if any changes are made. And I still have some more work to do (see the 'warnings' on the front page... that I will likely remove soon).

I have to say that the process was a lot easier than I expected. Even enough for me to consider submitting more articles or editing others. 

I'd love to hear your thoughts on this.