DoneJS Stream Apps
During this weekend at HackIllinois, I worked on implementing two new components of the canjs project with the canjs and donejs developers. These two components work as npm modules, and we started new repositories for them.
The goal of our projects are to make webpages load in a more efficient: we stream data so that the browser can render whatever is available as soon as possible and do not have to wait for the request to completely finish before performing the task. The two components I worked on is related to the newline delimited JSON (NDJSON) stream.
canjs/can-ndjson-stream
This component creates a readable JSON object stream from the a fetch response stream.
Descriptions
We did not exactly work on an issue. This is part of the donejs workflow and we are implementing this feature from scratch.
In developing this project, I mainly dealt with asynchronous tasks and readable streams, both of which are new, experimental web technologies. Fortunately I am somewhat familiar with them. It took me the most efforts to sort out the correct order of asynchronous executions and chain them correctly.
Our mentor/canjs maintainer, teammates and I worked in conjunction on this project. I was responsible for the main coding. The project is almost fully completed at the current version, tested and documented, available on npm. The linger issues are failures with the Travis CI systems.
Commits
Since the repository is new, I directly commited to master
canjs/can-connect-ndjson
This components works with can-connect to support observable lists with data from ndjson-stream: the list grows and notifies its observers as ndjson-stream streams more and more data.
Descriptions
We did not exactly work on an issue. This is part of the donejs workflow and we are implementing this feature from scratch.
In developing this project, I created an interface that connects a ndjson-stream to a DefinedList(observable list). This project depends on existing canjs code, so it took me a while to understanding how canjs handles the tasks and how internal calls are made.
Our mentor/canjs maintainer and I worked in conjunction on this project while other teammates are working on testing and documenting the can-ndjson-stream. I was responsible for the main coding as well as testing. The project is published to npm, but the documentation and CI testing is incomplete yet.
Commits
Since the repository is new, I directly commited to master
I’m giving a lecture about this today (12/11), 2-3.15pm in DCL 1320.
Thanks for your effort on this! Bianca from Bitovi wrote it up here: https://davidwalsh.name/streaming-data-fetch-ndjson