Mimi Onuoha is a Brooklyn-based artist and researcher investigating the social results of data collection and computational categorization. Her work uses code, writing, performance, and objects to explore missing data and the ways in which people are abstracted, represented, and classified.
Onuoha has been in residence at Eyebeam, the Data & Society Research Institute, Columbia University’s Tow Center, and the Royal College of Art. She has exhibited and presented workshops in festivals internationally, and in 2014 she was selected to be in the inaugural class of Fulbright-National Geographic Digital Storytelling Fellows. She was recently a visiting faculty member at Bennington College, and currently is teaching at NYU.
Traceroute is a useful tool for debugging connectivity problems, but have you ever thought of it as a storytelling medium? It can be!
In this talk, I’ll explain how routing and traceroute work, and then we’ll use those mechanisms creatively to tell our own human-readable story in the path our packets take to a machine. Even better, we’ll do it with software in the consumer cloud – no commercial routing gear or fancy network config needed!
You just got a text message from your friend, and it contains three emojis and a box with an X in it: 🍴🌙 ⌧. Is that an emoji, too? What were they trying to say? Are they currently eating dinner under the moonlight, or are they inviting you to have takeout tonight?
Later in the day, you’re scraping information from a website that happens to have characters in another language on it, and as you pull data into your terminal, you start to notice more little boxes with X’s in them. Or perhaps your terminal renders them as � or 🥡. What does this all mean? Why are these symbols following you around?
You’ve just encountered “notdef”! This is the official term for the “Not Defined” symbol that is shown when the typeface being used doesn’t contain the character requested. This talk aims to show you what you can do in the face of notdef, including how you can build a little tool to search your machine for fonts that support the characters you need to tell your story – whether you’re using a design app to make a poster, or writing a paper that contains information in a non-Latin language.
Remember isometric games, like SimCity 2000 and Q*bert? Real life sadly doesn’t look like Q*bert, and it’s mostly because parallel lines converge! Why does this happen? We’ll use simple geometric rules to explain how cameras turn light, a five-dimensional (!) spatial variable, into a pattern of pixels. With computers, we can algorithmically alter that pattern, and synthesize impossible perspectives from real data. (Including some inspired by your retro-gaming faves!) Finally, we’ll discuss how image formation models might change the way we think about the world around us!
During the 2015 floods in Chennai, India, a few of us built an application that could crowd-source real-time flood data. It was used widely by at least 600,000 people in Chennai and went on to become a beautiful civic visualisation. I’ll talk about the exact workflow and tools we used to build this map, and you will take away an understanding of the libraries you can use to create your own map-driven projects.
For many people, their first (and only) experience of static types is being continually told off by their compiler. With a little gentle encouragement, it is possible to mend this dysfunctional relationship and move towards having a conversation instead.
In this talk I’ll demonstrate typed holes, which are a feature of some languages that allow them to offer suggestions when you ask for help!
How do you compress the set of all possible books? What can we learn about data compression by trying to do this? We’ll cover zip bombs, the pigeon-hole principle, Kolmogorov complexity, and the Hutter Prize, all compressed into a ten-minute talk.
(Note: This talk was originally scheduled to appear at !!Con 2017.)
I was playing my old copy of Pokemon Blue for the original Game Boy one day during winter holidays, for old times’ sake. About an hour in, I realized that this was probably going to take me on the order of 20-30 hours to complete.
20-30 hours! Who has time for that?
What I’d really like to do is sit down, think very carefully, and write down all the moves I’d like to make, all at once, and then pass them to a regular expression, which will accept if (and only if) those moves will win a game of Pokemon Blue. This approach has a lot of advantages! For example, in addition to being faster to specify a complete moveset for a game, it allows you to put your entire game of Pokemon Blue into a version control system (e.g., git), which allows you to apply the “infrastructure as code” philosophy to your Pokemon Blue runthrough.
Inspired by environments like Overtone and Sonic Pi, I made Overscan, an open-source live-coding environment to explore ideas in video streaming and broadcasting. Overscan brings the expressive, improvisational power of a Scheme REPL to live video creation and compositing. I will live demo the debut of Overscan, creating a video stream while compositing graphics and text on the fly. I will peel back the layers of abstraction underlying Overscan to reveal its heart: the Racket programming language, the GStreamer multimedia framework, and the GObject Introspection library. This talk will amuse the littlest Schemer or the most grizzled C programmer, and in the end, you’ll be able to use code to bend video to your will. Let’s take back the airwaves! Or have a goof!
Inspired by Ramsey Nasser’s Arabic programming language Qalb, a functional, Lisp-style language, I created Noor, an Algol-based, imperative Arabic programming language. Arabic is my first language, and I wouldn’t have learned to program had I not moved to the US and learned English. I created Noor with the intention of it being a language for middle schoolers and high schoolers. In this talk I’ll share what I learned about creating an Arabic programming language and how Arabic text really works on your computer.
How can code that never runs still cause a program to crash? What does it mean to link a library into a compiled program? To answer these questions (and shed some light on the mysterious dark art of linking), we’ll methodically unravel a surprising edge case of static linking in a C++ program.
Over 1,500 human-made satellites are currently orbiting the Earth, and most of them are quite chatty! You might immediately think about the ones we use daily – for TV, GPS, or the internet – but there are hundreds of much smaller and lesser-known satellites that are just as deserving of our attention and love!
Let’s hear what they have to say! We’ll build an antenna out of spare coat hangers and use it to listen to the tiny amateur satellites telling stories about the world around us, playing music, delivering messages, or even speaking in lovely robotic voices!
Making games is hard. Making games with a limited number of sprites, sound effects, colors, bars of music, and even lines of code is… well… sorta mind-bending, actually. Because when you bump up against those restrictions you start to uncover secret trade-offs between these limited resources. Would I rather my tasty mushroom sprite have red spots, or would I prefer to be able to save high scores? How many notes in a song is a for-loop worth?
Come and learn about the beautiful Bizarro World that is fantasy console game development! And maybe, just maybe, you’ll take away some insights on programming in general.
The idea that there are undecidable problems – ones that simply cannot be solved with computers – is kind of mind-blowing. The most well-known is the halting problem: given a Turing machine (program), will it halt (terminate)? Many other undecidable problems are either very similar to the halting problem, or very abstract. In this talk I’ll show one that is neither! The Post Correspondence Problem is a word tile puzzle that is actually undecidable. I’ll walk through how to prove this, by building a special puzzle that has a solution if and only if a Turing machine would halt. At the end, I hope you’ll share some of my wonder at the theoretical side of computers!
David Burge described the internet as a worldwide cat picture distribution system. But historically humans have had to do all the work of taking those pictures ahead of time! I wanted to empower my three cats and one dog to take their own pictures and send them to me, but how do you design for cats and dogs?
I will describe how I built a home photo booth from a Raspberry Pi, various electronic components, the Twilio API, and a ton of cat treats to allow my pets to send me selfies.
In a direct challenge to the demo gods, this talk will end with an attempt at a live text exchange with my pets from the stage!
Your phone knows where you are at all times, usually with pretty shocking accuracy. How the heck does it know that?!
This talk will dive into the math and physics underlying modern smartphone location technology (GPS, AGPS, and indoor location), but we’ll also take a walk through the history of location tech. Modern GPS arose out of Cold War-era US DoD research, but how is that research connected to the larger historical and political context of geolocation throughout the ages? How is the technical solution to tracking Russian missiles directly indebted to the 17th century sailors who developed smarter ways to locate themselves on the open sea? This talk will answer all of that and more!
Building technical and leadership skills doesn’t only happen in the workplace! I became a better technical leader and Site Reliability Engineer from playing games such as Puzzle Pirates, World of Warcraft, EVE Online, and Factorio. I will share what I learned from these experiences, and how both hiring managers and employees can talk about non-traditional forms of experience.
If you thought getting datetimes and daylight savings correct was tough, wait ‘til you see what relativistic time looks like! In this talk, I’ll show you how keeping clocks in sync becomes literally impossible as soon as you start accelerating away from Earth, and what trying to account for this case might mean if we want our current implementations of datetime libraries to work correctly.
If you are like me and not very good at the game of darts, you might be able to use your skills towards estimating the value of pi. In this talk, we will look at a simple Monte Carlo algorithm which uses the (scaled) ratio of the number of darts that land on a dartboard over the number of darts that land within a square around the dartboard as an unbiased estimate for pi. We will understand why the algorithm works intuitively by appealing to the Law of Large Numbers. Furthermore, since we would like to not throw darts forever, we will use a large deviation inequality to derive an upper bound on the number of throws we need to estimate pi to within a desired accuracy with high probability.
According to Einstein’s theory of special relativity, space and time are not absolute, and objects become shorter when moving at larger velocities. So what happens if you take a photo of an object moving at, say, 90% of the speed of light? To answer this question, I’ll start with a normal ray-tracer used to render 2D views of 3D scenes, discuss the consequences of special relativity for moving objects, and then modify the ray-tracer to be consistent with special relativity. Even if you already know about special relativity, the answer might still be surprising.
I’ve always wanted to have conversations with past versions of myself, but I worried about the dangerous side effects of using time machines. With the emergence of deep learning and the seq2seq model, a new solution is available!
Using chat logs I’ve archived since early high school, I’ve trained one chat bot on messages from my teenage years, and paired it up with a bot trained on more recent messages. Hilarity will ensue as I algorithmically converse with a version of myself that’s younger, has significantly worse taste in music, and holds quirky political beliefs. We’ll also learn how easy it is to get started with machine learning, TensorFlow, and seq2seq!
Ever seen /proc in Linux? It’s the best-known “synthetic filesystem”, a set of “files” and “folders” on your computer that don’t actually map to anything on a real disk. Instead, /proc controls the processes on your computer.
Synthetic filesystems are a powerful and underused interface: there are tons of programming tools and commands to work with files, and once you have a synthetic filesystem, you can use all of them right away!
In this talk, I’ll go over four other synthetic filesystems: GrabFS, which exposes screenshots of how all your Mac windows look right now, YTFS, which exposes YouTube videos as video files on your computer, btfs, which lets you mount a BitTorrent file and lazily downloads the pieces you want for you, and WikipediaFS, where you can view and edit Wikipedia articles as files.
For each synthetic filesystem, I’ll show you one cool trick you get for free just because it’s a filesystem and not just a Web service or isolated program.
Finally, I’ll talk about some general advantages and disadvantages of this approach, which was pioneered by the Plan 9 OS in the early 1990s.
Star Wars is a movie, and Postgres is a database. And what better way to watch a movie than using only a database? I mean, aside from in a theater, or on a TV, or on a tiny airplane screen, or… So while I guess I can’t say it’s the best way, I can for sure say that it is a way.
Even if you don’t go and watch all your movies this way in the future, you’ll learn how to combine several unrelated features in unexpected ways, and that’s where all the fun comes from in programming!leinweiber
HQ Trivia is a game about what you know, what you can guess, and ultimately, luck. Twelve multiple-choice questions of increasing difficulty make it very unlikely that anyone will win by chance alone. And the questions get really obscure and hard towards the end! So what’s a player to do? Cheat!
I’ll get down to the nitty-gritty about how I wrote an iOS app (and later a MacOS app) that takes advantage of Apple’s Vision framework, open source OCR software, and plain ol’ Google to help me decide on my answers while playing HQ Trivia. I’ll also talk about why I haven’t won a single time (though I’ve gotten close), and why some problems are just really hard for software to solve.
This year marks the 50th anniversary of the recording of At Folsom Prison, a project representing one of the many ways Johnny Cash challenged commercial music’s conventional boundaries. Another example: Johnny Cash initially wanted to record religious music and managed to release a few early gospel tracks despite outright resistance from his first producer, Sam Phillips. Much later, Rick Rubin embraced Cash’s interest in religious themes and gave him broad creative license. While both producers intended to sell Cash to a rock audience, Rick Rubin had a more relaxed interpretation of rock’s relationship to gospel music and provided Cash much wider creative latitude.
This talk will focus on a pivotal song, “The Man Comes Around”. We’ll use high-level audio attributes derived from machine learning and spectrum analysis to measure the audible distance between it and gospel music on either side of Cash’s discography. Clustering the two groups with The Man as the fulcrum, we’ll visualize the extent to which modern music has reinterpreted the rock genre and expanded its sensibility.
Most computer algebra systems are good at solving equations, but aren’t very good at showing their work. If we’re looking to educate students and automatically explain a problem intuitively, how does our approach to developing these systems have to change?
Let’s talk about mathsteps, an open-source library that solves basic algebra problems and not only provides the answer, but shows you the steps that get you there. Some algorithms are designed to optimize for space or speed, but I’ll talk about how we design algorithms to optimize for teaching – algorithms that explain to you what you’re doing and why.
We’ll walk through the process of how a human might solve algebra problems and how that can be translated into an algorithm – from parsing math input, to using data structures that make teaching easier. Finally, we’ll talk about the future of tools like mathsteps – possibilities as well as limitations.
I spend my days typing out code in a text editor. But why do words and text get to have all the fun, if under the hood it’s just zeros and ones? This is a talk about some of the other fun, quirky, beautiful ways we could tell the computer how to do our bidding, from visual expressions of logic and spatial arrangements of symbols to manipulating physical objects in the real world.
I’ll start with a visual history of some of the major visual programming languages, with lots of pictures. Then we’ll talk through the ins and outs of some of the hairiest problems for visual languages, like how to do control flow and how to have abstraction and code reuse. Finally, we’ll talk about what it all means. What problems can visual programming solve better than any other paradigm? And are any of these super-creative visual idioms and ideas things we can pull back into our text-based programming worlds?
Running large, long-running background tasks on your system may have the unintended side effect of consuming all the resources you need to run whatever you had it for in the first place. In this talk we’ll recreate from scratch a function that will throttle your tasks to closely match available capacity, using methods created to control the speed of Dutch windmills during the Industrial Revolution.
The itty, bitty, tiny bytes that make up a Pokemon!
In 2013, Nintendo released Pokebank, an app that allowed players of the Pokemon game series to upload and store creatures called Pokemon from their games on Nintendo servers. Then it broke!
Pokemon are stored in a mere 232 bytes, yet players from Japan uploaded so many of them to Nintendo’s servers that the Pokebank app’s release was delayed by months in the US and Europe – but that’s not the only thing that’s interesting about the Pokemon data structure. Learn about what the data structure looks like and how it’s adapted over the last 22 years and over four different consoles!
Computers have their own aesthetics, which come from their limitations.
In this talk, I’ll show you a family of glitch art that is so fundamental to the idea of pixels that it has been independently rediscovered, repeatedly, ever since humans first started asking machines to print drawings on paper.
Every researcher who has found these has given them a new name – they’ve been called CIRCLE2 algorithms, dot-matrix holograms, degenerate Voronoi patterns, and others – but I think “pseudofractals” is the most evocative descriptor.