Two STEAM Examples

We present two real world examples of ways that STEM and the arts can be connected.

In the first, art museum professionals partner with medical educators to improve medical practice.

In the second, art gallery visitors are guided through an exhibition by A.I. (artificial intelligence technology).

Examining Art to Improve the Medical Examination.

In June of 2016 Bonnie Pitman recently retired as the Director of the Dallas Museum of Art, convened a major conference entitled “The Art of Examination: Art Museum and Medical School Partnerships.  Participants represented sixty art museums and their partner medical schools at New York’s Museum of Modern Art (MoMA).

Partnership between art museums and medical schools are part of an emerging field called medical humanities, an interdisciplinary field in which knowledge from the arts makes contributions to medical education and practice.

The conference served as a platform that “provided a sound overview of the fields’ best practices, goals, history, terminology, evaluation, and future directions.” Such partnerships at major art and medical institutions in the U.S. and abroad are advocates for such programs and build a bridge between the arts and sciences.” (Pitman, 2016)

Careful examination of the history, composition, themes presented by an art object marks the work of the art critic or art educator.

Similarly, a physician begins her work with an examination of the patient’s various physical and affective characteristics, some of which may be important to the diagnosis while others are not. The ability to discriminate between the meaningful from the inconsequential is therefore an important skill shared by the art educator and the physician.

The first such art museum-medical school partnerships was created in 1999 when Dr. Irwin Braverman (Yale Medical School) and Linda Friedlaender, (Yale’s British Art Collection) began to work together to develop the observational skills of medical students by training them to use the techniques and language of art criticism as they learned how to examine their patients. (Pitman, 2016)

The connections between art and medical practice have led to at least one hundred such partnerships currently. There were sixty at the conference from U.S. Canada, England, and Italy. Forty more were on a waiting list for the conference.

A growing body of research literature published in medical journals also attests to the power of the intersection of where art museum and art professionals work with medical educators to the benefit of both health care professionals as well as to the community at large.

Going to the Tate Britain with an (artificially) Intelligent computer program named Recognition

You can visit the Tate Britain in London in person or online to both see and interact with the exhibition called Recognition. (Do not delay: Recognition closes on November 27)

The development of the exhibition was stimulated by the offer of the 2016 IK Prize that offers incentives to promote the use of digital technology in the arts.

The Tate Britain’s “mission is to increase the public’s enjoyment and understanding of British art from the 16th century to the present day…” as well as to increase the numbers of people who come to view the art; particularly young millennials whom it is hoped will become the next generation of art lovers.

However, Tony Guillan of the Tate recognized that looking at art and seeing art are not necessarily the same, the difference being that looking is simple discrimination, “that’s a painting,” while seeing connects the art to reality.

The successful quest for the IK prize began with the insight that the project would use A.I. technology, “…because getting machines to do what humans can do is one of the most exciting frontiers in technology…Is there anything more human than looking at art?” (Dobrzynski, 2016)

To compete, Tate Britain enlisted a number of partners: Microsoft, JoliBrain, a French A.I. company, and  Fabrica, an Italian communication research company. Fabrica would lead the development of Tate’s entry.

The team at Fabrica began with the question: “What if we could link our everyday lives to the Tate’s collection to illuminate similarities between the present and the past?” They developed the idea that the goal could be met by allowing the viewers to “see the world through two different lenses,” how the world has been represented historically by artists and how the world is represented today, through the news media.  (Dobrzynski, 2016)

Under Fabrica’s leadership, the partners began to work: Microsoft provided programming support, JoliBrain  contributed their DeepDetect API (application programming interface, a set of routines, protocols, etc. that makes it easier to develop programs) as well as DeepDetect server where the program would be run.

Fabrica put a variety of artificial intelligence technologies together, “including computer vision capabilities, such as object recognition, facial recognition, colour and composition analysis; and natural language analysis; and natural language processing of text associated with images, allowing it to analyze context and subject matter and produce written description of the images comparisons.” (

As Recognition (or [re] [cognition]) works it creates a virtual collection of images by matching works from the Tate Britain collection with contemporary news photos from the news agency Reuters. The matches are based on similarities of objects, faces, composition, theme that the A.I. finds as it views images.

The human viewer can click to stop the process in order to examine any of the matches in the virtual gallery in order to give Recognition feedback by responding to the prompt: “what makes this an interesting match?”

A.I. has been used in health care and transportation but A.I. in art is “uncharted space” according to Microsoft’s Eric Horovitz which is why Microsoft was interested in working with the project. It is an opportunity to see how A.I. can be “creative and make mistakes and meander.” (Dobrzynski, 2016)

The humanities and science and technology are often seen as separate worlds; the one supposedly subjective, intuitive, vague; the other, objective, precise fact-filled.

But perhaps not. A medical student constructing her examination of a patient using language and insights from art criticism; Science? Art?

Human art gallery visitors are given a tour by an A.I. program that shows works from the gallery matched with news photos.

The humans are asked for their assessment of the match. The program uses the human generated assessments to refine its matches.

Humans and machine learn from one another. Humanities? Art? Technology? Science?

Time to reasses our categories.




Dobrzynski, J. H. (2016c). Artificial Intelligence as a Bridge for Art and Reality. New York Times, p. 18. Retrieved from

Sheets, H. M. (2016c). How an Aesthete’s Eye Can Help a Doctor’s Hand

New York Times. Retrieved from


Pitman, B. (2016). The Art of Examination: Art Museum and Medical School Parnerships. Proceedings from The Art of Examination: Art Museum and Medical School Parnerships, New York and Dallas.


STEM, STEAM, art museum-medical school partnerships, clinical practice, A.I., Tate Britain, human assessment, A.I. and art


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s