Building a TCP API: How we built our realtime app, Filmgrain, part 2
In this technical retrospective (2013), @Guzba shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
Core Principles for Independent Builders:
Defaulting to simpler architectures (such as monolithic frameworks, SQLite, or managed VPS) prevents early engineering bankruptcy.
Evaluating true compute and egress costs reveals that high cloud bills can often be slashed by 80%+ through judicious re-platforming.
Code you write at 2 AM as a solo engineer must be easy to debug six months later without complex microservice dependencies.
Community Debate Themes:
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
CATEGORY Frugal Engineering →
HISTORICAL ERA Web 2.0 Genesis (2007–2013)
COMMUNITY SCORE 18 points
TOTAL DISCUSSIONS 20 comments
Props for trying a new approach, Just Because. Here's some feedback:
Until someone decides to inject a newline into your JSON, then you've got a problem [1]. Just use a length-prefixed encoding, there are tons: bencode, tnetstrings, etc. Also, JSON objects delimit theselves, although you could use a proper stream parser to handle fragmented messages better. A secondary transport like websockets (as others have mentioned) would also give you framing for free.
Just use DNS. You're already making a DNS request for S3, so here you have a redundant HTTP request, and leak network-level operational details in application logic.
[1] Ignoring HTTP-style newlines here for moment; common implementations are battle-hardened, but it doesn't mean newline-terminated protocols are really a good idea...
After ages fighting slowness and lag on an HTTP-based API for some hardware (Hue lights) that could save 98% of its bandwidth using a TCP interface, I was thinking just last week that HTTP obsession is getting out of hand, and that I'd like to see more developers using TCP directly. Good on you.
HTTP lets you inherit a large and diverse body of infrastructure - load balancing, monitoring, testing, logging, etc etc.
Also, some combination of websockets and/or keep-alive would yield most of the benefits of using just TCP.
I get that there's more than request-response. Why not use Websockets instead? What do you plan to do if your mobile users want a browser-based experience?
Sounds like you re-invented the wheel.
Google Cloud Messaging and (to a lesser extent) websockets would have been better solutions.