twitter - Streaming API vs Rest API? -

the canonical example here twitter's api. understand conceptually how rest api works, query server particular request in receive response (json, xml, etc), great.

however i'm not sure how streaming api works behind scenes. understand how consume it. example twitter listen response. response listen data , in tweets come in chunks. build chunks in string buffer , wait line feed signifies end of tweet. doing make work?

let's had bunch of data , wanted setup streaming api locally other people on net consume (just twitter). how done, technologies? node js handle? i'm trying wrap head around doing make thing work.

twitter's stream api it's long-running request that's left open, data pushed , when becomes available.

the repercussion of server have able deal lots of concurrent open http connections (one per client). lot of existing servers don't manage well, example java servlet engines assign 1 thread per request can (a) quite expensive , (b) hits normal max-threads setting , prevents subsequent connections.

as guessed node.js model fits idea of streaming connection better servlet model does. both requests , responses exposed streams in node.js, don't occupy entire thread or process, means continue pushing data stream long remained open without tying excessive resources (although subjective). in theory have lot of concurrent open responses connected single process , write each 1 when necessary.

if haven't looked @ http docs node.js might useful.

i'd take @ technoweenie's twitter client see consumer end of api looks node.js, the stream() function in particular.