nsq-to-gs – Streaming NSQ messages directly to Google Cloud Storage

In addition to my previously published (very early) project to stream NSQ messages directly to BigQuery, I am happy to presents a modified version of nsq-to-s3 that supports streaming NSQ messages directly Google Cloud Storage. Grab it while its hot from the nsq-to-gs repo. I do see a future for a merged version of these two projects that supports both S3 and Google Cloud Storage but this would have to be enough for now. [Read More]

nsq-to-bigquery – Stream messages from NSQ directly to Google BigQuery

In the spirit of nsq-to-XXX such as nsq-to-http and nsq-to-file – I bring you the very first version of nsq-to-bigquery. nsq-to-bigquery, as the name suggest, streams data from an NSQ channel into Google’s BigQuery using the Streaming API and provide very effective means to stream data that should be then further analysed and aggregated by BigQuery’s excellent performance. This is a (very) initial version so it has some limitations and assumptions. [Read More]