Jesse Martinez

jessejm [at] cs [dot] washington [dot] edu

I am a fourth-year PhD Student at the Paul G. Allen School of Computer Science & Engineering at the University of Washington.

I do research in Human-Computer Interaction with James Fogarty, focusing on Interactive Media Accessibility. I completed my BS in 2019 at the University of Chicago, where I did research with Blase Ur and worked with Patrick Jagoda on the 2017 ARG, the parasite.

I'm also a computational artist, transmedia game designer, puzzle creator, and tabletop & video gamer.

Email  /  CV  /  Google Scholar  /  GitLab

profile photo
Publications

[ PDF ] A Preliminary Analysis of Android Educational Game Accessibility
Jesse J. Martinez, James Fogarty, Jon Froehlich
Extended Abstract Proceedings of ASSETS 2021.

[ PDF ] Joint Media Engagement in Families Playing Animal Crossing: New Horizons During the COVID-19 Pandemic
Jesse J. Martinez, Travis Windleharth, Qisheng Li, Arpita Bhattacharya, Jason C. Yip, Katy E. Pearce, Jin Ha Lee
CSCW 2022: Proceedings of the 2022 ACM Conference on Computer Supported Cooperative Work and Social Computing

[ PDF ] "I just need to have a couple White Claws and play Animal Crossing tonight": Parents coping with video games during the COVID-19 pandemic
Katy E. Pearce, Jason C. Yip, Jin Ha Lee, Jesse J. Martinez, Travis Windleharth, Qisheng Li, Arpita Bhattacharya
Psychology of Popular Media 2021

[ PDF ] Families Playing Animal Crossing together: Coping with video games during the COVID-19 pandemic
Katy E. Pearce, Jason C. Yip, Jin Ha Lee, Jesse J. Martinez, Travis Windleharth, Qisheng Li, Arpita Bhattacharya
Games and Media 2021

[ PDF ] When Smart Devices Are Stupid: Negative Experiences Using Home Smart Devices
Weijia He, Jesse Martinez, Roshni Padhi, Lefan Zhang, Blase Ur
Proceedings of the IEEE Workshop on the Internet of Safe Things (SafeThings).
San Francisco, CA, May 2019

[ PDF ] AutoTap: Synthesizing and Repairing Trigger-Action Programs Using LTL Properties
Lefan Zhang, Weijia He, Jesse Martinez, Noah Brackenbury, Shan Lu, Blase Ur
Proceedings of the 41st ACM/IEEE International Conference on Software Engineering (ICSE).
Montreal, QC, Canada, May 2019.


Current Projects
Accessibility & Game Adoption   ◆   Research

Project Advisors: James Fogarty, Jon Froehlich

My primary research explores the current strategies gamers with disabilities use during the game adoption process, to advise game developers, publishers, and distributors on how they can better support these practices. This work seeks to address research questions across the three main phases of the "game adoption process":

  • Game Discovery. What resources currently exist to support gamers with disabilities in discovering games to play, and how can we support access needs in this interaction?
  • Game Evaluation. How are gamers with disabilities supported (or not supported) in evaluating whether a game meets their access needs, and how can game publishers and distributors better accommodate players' existing evaluation methods?
  • Game Adaptation. How can we support gamers with disabilities in adapting inaccessible games to meet their access needs? What strategies are currently used, and how can these strategies inform the design of tools to make the adaptation process easier?
Accessible Data Visualization Experiences   ◆   Research

Project Collaborators: John Thompson, Bongshin Lee, Alper Sarikaya, Ed Cutrell

In this project, we aim to design and develop a system that enables Blind & Low Vision Screen Reader Users to meaningfully interact with web-based data visualizations using an off-the-shelf screen reader, with a particular emphasis on exposing the communicative intent and functionality of a visualization. To design such a tool, we use co-design methods with a team of BLV individuals, and iteratively design, prototype, & evaluate our tool in collaboration with our design partners.

This project is a collaboration with the MSR EPIC & Ability teams, and was part of my 2022 summer internship work.

Personalized and "Unprofessional" Audio Descriptions   ◆   Research

Project Collaborators: Kelly Mack, Emma McDonnell

Building on the current practices people with disabilities and their friends use to describe video content, can we find a way to improve audio descriptions to be better tailored to the type of content the listener wants described?

We use the term "unprofessional" lovingly here: much "professional" audio description strives to provide an objective, unbiased lens, which is very effective and important in some contexts! But a movie night with friends doesn't need to be "professional," and people who rely on audio descriptions should have the option to let loose and be a bit "unprofessional" every now and then.

Repairing Android Game Inaccessibility   ◆   Research

Project Advisors: James Fogarty, Jake Wobbrock

Project Collaborators: Mingyuan Zhong

This project involves identifying, characterizing, and repairing accessibility barriers in Android games. This work seeks to address many different research questions, including:

  • How can game developers & publishers better support gamers with disabilities in evaluating the accessibility of games?
  • What is the current state of accessibility in the space of Android educational games?
  • How can we design and build accessible, dynamically-generated interfaces to help users overcome accessibility barriers?

Publications

[ PDF ] A Preliminary Analysis of Android Educational Game Accessibility
Jesse J. Martinez, James Fogarty, Jon Froehlich
Extended Abstract Proceedings of ASSETS 2021.

Designing an Accessible Tabletop Game   ◆   Research

Project Collaborators: Carol Mertz, Morgan Baker

This project, initiated through a collaboration on Exploding Kittens's Game Hand-to-Hand Wombat, seeks to explore how game designers and industry professionals can better incorporate accessibility considerations into their existing design and development practices.

Sudoku Impressionism   ◇   Creative Project

Inspired by Rubik's Cube Mosaics by artists like Giovanni Contardi, I'm exploring whether another common puzzle--the sudoku--can be used as a medium to create art.

The idea is simple (at least in my head): first, you take a completed sudoku grid and color each square in according to a specific color key (all 1's are color A, all 2's are color B, and so on). Then, tesselate multiple grids together to create a completed piece!

This project is still in early stages, requiring way more complex math than I expected, and honestly I'm not even sure if it's gonna work, but stay tuned!

"JPEGson Pollock"   ◇   Creative Project

This is a computational art project that I've been working on for several years, in which I've created a tool to take the input pixels of an digital image, turn them into "balls of paint" in a constrained 3D space, and the SPLAT them onto the "canvas" and output the result.

The project is definitely inspired by the work of Jackson Pollock, but oddly enough, his pieces produce some of the least interesting results (imo).


[ Imgur Gallery ]


Past Projects
Animal Crossing, Families, and COVID-19   ◆   Research

Project Advisors: Jin Ha Lee, Katy E. Pearce, Jason C. Yip

In this project, we are working to develop & propose design guidelines for game designers and developers to support collaborative play in families and coping with global crisis, based on interviews about engagement with Animal Crossing: New Horizons during the COVID-19 pandemic.


Publications

[ PDF ] Joint Media Engagement in Families Playing Animal Crossing: New Horizons During the COVID-19 Pandemic
Jesse J. Martinez, Travis Windleharth, Qisheng Li, Arpita Bhattacharya, Jason C. Yip, Katy E. Pearce, Jin Ha Lee
CSCW 2022: Proceedings of the 2022 ACM Conference on Computer Supported Cooperative Work and Social Computing

[ PDF ] "I just need to have a couple White Claws and play Animal Crossing tonight": Parents coping with video games during the COVID-19 pandemic
Katy E. Pearce, Jason C. Yip, Jin Ha Lee, Jesse J. Martinez, Travis Windleharth, Qisheng Li, Arpita Bhattacharya
Psychology of Popular Media 2021

[ PDF ] Families Playing Animal Crossing together: Coping with video games during the COVID-19 pandemic
Katy E. Pearce, Jason C. Yip, Jin Ha Lee, Jesse J. Martinez, Travis Windleharth, Qisheng Li, Arpita Bhattacharya
Games and Culture 2021

"Another Arrow to the Knee: Alternatives to Town Guards in Fantasy RPG's"   ◇   Creative Project

This speculative design RPG module was developed for Coda Game Collective's 2020 "Post-Police Design Challenge" Game Jam. The module seeks to question the ubiquity of police & policing in fantasy games (most frequently manifested as "Town Guards"), characterize the ways in which these are used, and provide alternatives to help RPG worldbuilders to imagine and create police-free worlds.


[ itch.io Link ]

End-User Automation of Smart Home Devices through Trigger-Action Programming   ◆   Research

This project involved enabling and empowering non-technical end-users of smart-hom devices to automate their devices by creating rules and constraints with Trigger-Action Programming (TAP). In my time on the project, I contributed to the design of the system, developed the back-end of our end-user web app, and helped run & analyze the results of a user study.


Publications

[ PDF ] When Smart Devices Are Stupid: Negative Experiences Using Home Smart Devices
Weijia He, Jesse Martinez, Roshni Padhi, Lefan Zhang, Blase Ur
Proceedings of the IEEE Workshop on the Internet of Safe Things (SafeThings).
San Francisco, CA, May 2019

[ PDF ] AutoTap: Synthesizing and Repairing Trigger-Action Programs Using LTL Properties
Lefan Zhang, Weijia He, Jesse Martinez, Noah Brackenbury, Shan Lu, Blase Ur
Proceedings of the 41st ACM/IEEE International Conference on Software Engineering (ICSE).
Montreal, QC, Canada, May 2019.

the parasite (2017)   ◈   Research / Creative Project

An Alternate Reality Game (ARG) in which incoming first-year students at the University of Chicago worked together to unravel the mystery of a secretive organization known only as "PS".

This transmedia game & research project was a collaboration with a large interdisciplinary team; full design credits can be found on the Project Archives (linked below).


[Project Archives | Wired article | UChicago Magazine article & photo gallery]

page layout credits: Jon Barron