The #DataOpen Journey

I’ve never been so happy sleeping so little. For five days this November, I taxed my mind and my laptop crunching through education policy data on the way to submitting an entry in the 2017 Data Open Championships, organized by Correlation One and Citadel.

The local datathons had been 7-hour sprints, where we rode a tidal wave of adrenaline to victory. To give teams a better chance to show their best, the Championship round would be a 5-day window instead- giving us more time to look for gems, but also more time to get lost in rabbit holes. Team dynamics would also be upended; would we work well when deadlines weren’t so pressing?

We kept ourselves to a schedule- though I poked fun at how Soren would announce a strict deadline, then push it back twice so that he could finish his part of the task. As we could afford spending twenty minutes debating the best approach, this balance between deadlines and flexibility helped manage our stress levels- enough pressure to keep us focused, but not so much that we became frazzled and panicked.

  • Two hours after the kick-off, we had narrowed down to studying primary/secondary school policy, and had a shortlist of potential topics.
  • Day 1 was spent exploring potential topics, pulling in additional datasets, and assessing whether we had enough data to identify potential topics.
  • By midday of Day 2, we knew had chosen to study charter schools, and had shut down our analyses on other topics.
  • Day 3, we promised ourselves we would do all of the district-level assessments. We almost got there- but significant parts were still left dangling.
  • Day 4 was spent with school-level assessments, literature review (my forte), and beginning to write the report.
  • Day 5 was a mad sprint of pulling together the final report, refining figures, and creating the slide deck. And then suddenly, all at once, it was over and we could tidy up, grab a beer, and let weariness overtake us.

Overall, the team dynamics were fantastic- we checked in often, debated contentious points without becoming stubborn, and learned from each others’ expertise- particularly reassuring as I came from a very different background than the others. Everyone was ready to help out the team, but happy to defer to the person best-suited for the task.

We didn’t sleep much, but we laughed a lot- at the little mistakes we made along the way, at the funny work habits that we each had, and at the ludicrous nature of the situation. I fooled myself with a case of “Happy Thanksgiving!” cupcakes which I had found discarded outside the building, pizza from departmental events, and snacks from another hackathon happening at the Berkeley Institute of Data Science.

The team, smiling amongst the empty pizza boxes and burrito wrappers in the conference room we commandeered. The room stank.

At the end of the five days we had a polished report that we were truly proud of, a stack of pizza boxes and burrito wrappers that threatened to fall off our conference table, and happy but exhausted grins on our faces. No matter what the judges might decide, we’d had a great experience working with an incredible team, and a research project that we could be proud of.

Leave a Reply

Your email address will not be published.