All credit go to Alexey Nezhdanov <snake@penza-gsm.ru>, I just ended up re-implementing his idea.
Alexey Nezhdanov sent a patch that made git-daemon usable from inetd (ie where inetd has already done the accept on the new connection, the fork, and the setup of stdin/stdout). I wanted to organize the thing slightly differently, though.
.. and does a "git-upload-pack" on demand.