Is it possible to program chips like the Arduino uno's Atmega328P directly with USB serial from your computer?
Put a bootloader on the chip first, then an offboard USB/Serial adapter can be used for the serial programming. That's basically what a ProMini is - a bootloaded '328P (in SMD form) with a resonator & decoupling caps and a small voltage regulator - and a header to accept an FTDI Basic for programming serially. If you notice the Black and Green colors next to the FTDI header, those are to match the wire colors on the the FTDI cables that were originally used, before seperate USB cable and FTDI modules became more popular.
Instead of the 6-pin FTDI header, you could make an adapter board to go from FTDI to a USB-A 5-pin connector, then on your board have a USB-B-Mini or USB-B-Micro 5-pin connector to bring in the FTDI's 5V, Gnd, DTR, Tx, and Rx signals.
USB wouldn't be coming in directly, but you would have the spacing advantage of the small USB connector, and could program serially from a PC USB port.
Too bad you don't want to use the Arduino IDE, there are supported core codes to add to the IDE for the Attiny85 and other chips like that. Attiny85 with bootloader could then be serially programmed with an offboard USB/Serial adapter.
This core supports the following processors - essentially every ATtiny processor that makes sense to use with Arduino. Click the processor name for part-specific information:
ATtiny24, 44, 84 (With or without Optiboot bootloader)
ATtiny25, 45, 85 (With or without Optiboot bootloader)
ATtiny261, 461, 861 (With or without Optiboot bootloader)
ATTiny87, 167 (with or without Optiboot bootloader)
ATTiny48, 88 (With or without Optiboot bootloader)
ATTiny441, 841 (With or without Optiboot bootloader)
ATTiny1634 (With or without Optiboot bootloader)
ATTiny828 (With or without Optiboot bootloader)
ATtiny43 (no bootloader)