Much of my technical work has been to support the construction and deployment of the reading devices I am studying. It has ranged from low-level electronic design, to device driver / kernel programming, to application software and UI development, and to distributed cloud-based applications.
Here is a sampling of the projects I have undertaken:
Cloud Infrastructure for Reading Devices
When moving our system out of the lab, into the real world, we needed a way to link together user slates with each other and with the user’s PC with minimal configuration and fuss on the user’s part. The system had been using a Bluetooth Personal Area Network (PAN), which required a special master device be present at all times, and required complex pairing procedures in order to add additional devices, like a user’s PC, to the PAN. Also, we needed a way to synchronize document annotations across devices, even if certain devices were offline at certain points in time.
Results
I deployed a cloud-based infrastructure for connecting together different reading devices. The system is a web application deployed through Google’s App Engine. Using a standard REST API over HTTP, slates and PCs can update the system about new annotations and other state changes. An XMPP connection to the App Engine server enables push-notifications to be sent to the connected devices.
User documents are stored in Dropbox, which allows slates fast and seamless access to documents that reside on the PC. For document manipulation/annotation tasks, the App Engine application pipes requests to a backend server on the Amazon Elastic Compute Cloud (EC2). This backend server downloads documents from the user’s Dropbox, applies changes, and saves the document out. In this way, synchronization conflicts across devices are minimized.
Some of the benefits this cloud-based infrastructure provides include:
- Device interchangeability. No need for “special” devices that always need to be present. Users have much more flexibility about what devices they use.
- High-availability. Annotations are always captured regardless of the device being used.
- Compatible with most networking setups. Since most network setups allow web access, devices behind firewalls can continue to communicate.
- REST API ensures that the system is device agnostic. Most devices, especially commercial tablets like the iPad, can easily open up an HTTP connection.
Capacitive Touch-Based Document Navigation
One of the interactional challenges we faced with the slates we created was that buttons were only minimally expressive. Standard tactile switches offered an up state and down state, which could be modulated over time (e.g. double-click, long press, etc.). We wanted a control scheme that additionally offered continuous adjustment (to flip a variable number of pages, or to make an absolute jump to a specific section of the document).
Results
I designed an input module using a Flex PCB and a Cypress CapSense chip. The input module fit into the tight space constraints of the slate enclosure and combines several capacitive touch sensing regions with tactile switches. Since all components were on a single Flex PCB, installation into the device was greatly simplified.
The combination of capacitive elements with tactile switches allows for continuous value adjustments along with tactile confirmation of input events. The system also enables richer interactions, like having buttons that respond differently depending on where they are pressed, as well as gesture-based interactions on the surface of the button.
Designing and building the input module consisted of creating rigid PCB prototype, on which I could verify that the electrode design would work, as well as test the initial versions of the microcontroller firmware. Once the flex boards were assembled, the rigid PCB doubled as the programming interface for the flex boards.
The final modules are in regular use in the custom reading slates being used for my lab studies and deployments.
Multi-Slate Reading System
While reviewing the requirements for a reading device suitable for work-related reading, we realized a viable reading solution needed to be thin and light, accept free-form ink annotations, and offer extended untethered operation. Moreover, to test our ideas about multi-display use, the devices needed to be able to communicate wirelessly.
It was quite clear that existing devices only supported subsets of these requirements. TabletPCs offered support for writing, but were generally bulky and could last only a couple hours on a charge. Other slate devices had smaller screens that forced users to scroll while viewing a letter-sized page at full size. Commercial e-book readers offered great readability and battery lives but limited support for writing. Most of these reading appliances also run proprietary software stacks, making developing custom software difficult.
Results
I worked alongside engineers at an industrial partner to develop a brand new electronic reading device that matched the requirements we identified. In creating the device I was responsible for the design of a number of electronic subsystems, and the entirety of the application software running on the hardware.
For the electronics, I was developed the power/battery subsystem, the interface between the Gumstix computer-on-module (COM) and the E-paper display hardware, and the interface between the COM and input microcontroller. For each of these subsystems, I created prototype PCBs to test circuit functionality before performing the final schematic capture.
For the application software on these devices, I developed a virtual framebuffer driver so that a standard graphics toolkit could be used with the specialized E-paper display drivers. To speed up my application development, I created a PC-based simulator to more quickly debug and iterate on my code.
Dual Display E-Book
Ethnographers studying reading had suggested that dual-screen reading devices would convey many of the benefits associated with reading on paper, with one of the major advantages being more effortless navigation. At that time, commercially available reading devices consisted of a single screen.
Results
We created several iterations of a dual-screen electronic reading device. Ranging from a technology demo, which showed how two independent computers could exchange sensor information in order to behave as a single unit, to a medium-fidelity prototype that was fully functional, albeit slightly larger than our target size and with limited battery life, to a final high fidelity unit that we used in user studies.
For all of these prototypes, I designed the electronics, and created the software for the devices. The tasks involved in the construction included making and assembling PCBs, low-level embedded systems development (for sensors and communications) and GUI programming (for the user interface).
To build the hardware, I made use of a number of rapid prototyping technologies, including 3D printing and laser cut parts. To reach our size and weight targets, I researched and refined process to perform low-cost SMT assembly using repurposed tools that can be acquired at stores like Home Depot and Target. Many of these techniques continue to be used in our lab, as well as those of our collaborators, for performing research with microelectronic devices.