Homework-solving robot launches SEAS students to PennApps victory
Instead of doing homework this past weekend, Christopher Yan, SEAS '15, and Derek He, SEAS '16, built a robot that could do math homework—and even write it out in their own handwriting.
Yan and He's project, Homework Help, won the first prize at this year's PennApps, one of the largest student hackathons in the world held at the University of Pennsylvania.
The robot uses a camera equipped with computer vision to identify simple math problems, solves them, and then mechanically writes out the answer with a pen.
He said the inspiration for the project came from his experience last year at the hackNY student hackaton, where fellow Application Development Initiative member Zack Newman, CC '15, won the top prize with a piece of software that solved jigsaw puzzles.
The robot works by measuring the distances between characters to determine whether they are part of the same number, and the space beneath them to figure out where to write the answer. Right now, the robot can only solve simple addition problems.
"How do we understand where the questions are? Clearly 85 plus 12 is not necessarily a hard question to answer, the real difficulty is understanding that eight and that five means 85," He said.
Yan described the robot as similar to a printer, but one that uses a motor and pen to write out answers instead of printing pixels.
"In order to actually do homework we actually mimic strokes," Yan said.
During the hackaton, Yan and He worked independently on different aspects of the machine and only came together at the last moment to combine the separate components.
"It was like 4 a.m. We weren't sure if we were going to finish on time, but we finally
connected it end to end," He said.
When they finally put everything together, the robot didn't actually produce a legible result. He and Yan eventually realized that it was writing the right answer but on the wrong axis.
"We realized that it was the right answer to the homework, printed with the axes flipped—we couldn't read it—and we realized, yeah this is it, we're actually going to finish," He said.
After meeting at last year's PennApps, He and Yan said they were able to form a successful team because they both lived in the ADI Special Interest Community this year.
"We probably wouldn't talk about this project if we didn't live near each other," Yan said. "We also have equipment we can borrow to use, and preparing all by ourselves would be burdensome."
He and Yan will now strengthen the "hand" of their robot—it is currently made out of plastic knives, flyers, and duct tape—before showing it off again at a demonstration day organized by ADI, Tech@NYU, and Create@Cooper at Cooper Union on Saturday.
"We're probably going to go there and show off to the broader community," He said. "It was a fun weekend project. I don't think we're going to try and build anything huge out of this. We got what we wanted out of it."
Though He and Yan said they don't think they're going to further expand on the project, they said they've gained a level of personal fulfillment.
"I was saying a week before PennApps, I'd love to win a hackathon before I graduated, as a college bucket list thing," He said. "There's a lot of personal satisfaction from making something I'm really proud of. I think it's really cool."
firstname.lastname@example.org | @ezactron