As you've worked out, obviously, your computer does have to be able to store 0s. You can either use a different symbol for 0, or you can use the lack of a 1 to store a 0 - but in that case, the computer has to know where the 1's could be so that it can tell between a 0, and something that's not actually part of the data.
In electronic signals, we can use a voltage for 1 and no voltage for 0. Key to this is that the computer is only looking for voltage on the signal wires. There's no voltage between the wires, or on the plastic case, but it's not a problem because the computer isn't looking there.
In the simplest SRAM memory cell, for example, you have two gates - one of which is on, and one of which is off. You pick one side and say "this is the side that stores the bit". If that side is on it's a 1, if that side is off it's a 0. There's no other possibility (both gates can't be off, unless the computer is unplugged) and there's no possibility of looking in the wrong place (the computer isn't going to miswire itself and try to read from where there isn't a memory cell). So there's basically no possibility for a screwup of this sort.
On tape and disk media, and serial signals in general, the computer does need to be careful to look where there is data. There are a whole lot of different approaches to this.
Perhaps the simplest is to just have two lines of holes on your paper tape, and say that a hole on the left side is a 1 and a hole on the right side is a 0.
More complicated encoding methods allow the computer to "lock onto" the hole spacing. If you have a hole every half centimetre, and you go half a centimetre without seeing a hole, that's a 0. But what if your computer measures it as 0.499cm instead of 0.500? Or what if the person who wrote the tape measured it as 0.499cm? Then it could drift out of sync and you'd see an extra 0 occasionally, or miss a 0. Therefore there needs to be at least one hole every so often, so the computer can reset its measurement so it doesn't get too far out of sync. The advantage of these kinds of methods, compared to the two-track method, is you can get almost twice as much data in the same space. (You could also go by timing, instead of distance)
In the UART protocol, you have a bit that is always 1, in front of each byte, and the 8 bits in the byte are timed. The 1 bit (which would be a hole in your case) gets the reader in sync for the next 8 bits (at least) and the reader knows not to send the extra bit to the computer.
Another approach could be to use a kind of frequency modulation - where (say) a hole less than 0.5cm from the last hole is a 1, and a hole more than 0.5cm from the last hole is a 0.