When the AVGN released his episode on Tiger Electronic games a few years back, the detail that stuck with me the most was that their GameCon console could be hooked up to a modem and access the internet (text only). That surprised me because I remember being able to do just that with my first cellphones through WAP, but that was much later in life (and still not that great).
Still, I did some shenanigans with phones and experimented a bit with both the PS2 and the Dreamcast, accomplishing little more than murdering my parents' phone bill in the process. it was mind-blowing.
What was the first console (or devise) that you took online? And what did you do with it?