Eric Bergman-Terrell's Blog

Node.js Programming Tip: Use Async to Organize Concurrent (and Serial) Tasks
August 26, 2015

The controller for this website's blog page retrieves the following data from the database, before rendering the page:

Previously, the controller had some slightly complex, nested code to retrieve all the above data. Even worse, the code did each retrieval serially, rather than in parallel.

In reality, once the N most recent posts have been retrieved, all the other retrievals can be performed in parallel. Thanks to the async module, I was able to restructure my ugly, serial code into a more elegant, parallel solution.

The processGetBlog function first retrieves the N most recent posts. This has to be done serially. But once that data has been retrieved, all the other retrievals are performed in parallel, using async's async.parallel function.

Once all the parallel retrievals have completed, or any have failed, the callback at the end of the async.parallel call is called. If there were no errors, the results parameter will be an array of all the results of the parallel tasks, in the order that the tasks appear in the async.parallel call. In other words, results[0] will contain the blog post data, results[1] will contain the previous and next blog ids, and results[2] will contain the blog comments. The order in which the parallel tasks complete does not affect the order of the results.

For situations that require retrieving multiple data items, I recommend that you try out async. Even if the data retrievals cannot be parallelized, async has other mechanisms that may work well, for instance, async.series.

function processGetBlog(req, res) {
    blogModel.retrieveMostRecentBlogPosts(7, function(error, mostRecentPosts) {
        var id = req.params.id != null ? req.params.id : mostRecentPosts[0].id;

        async.parallel([
            function(callback) {
                blogModel.retrievePost(id, function(error, blogPost) {
                    callback(error, blogPost);
                });
            },
            function(callback) {
                blogModel.retrievePrevNextPosts(id, function(error, prevNextPosts) {
                    callback(error, prevNextPosts);
                });
            },
            function(callback) {
                blogCommentModel.retrieveBlogComments(id, function(error, blogComments) {
                    callback(error, blogComments);
                });
            }
        ],
        function(error, results) {
            if (!error) {
                res.render('../views/blog/blog',
                    {
                        title: 'Eric Bergman-Terrell\'s Blog',
                        mostRecentPosts: mostRecentPosts,
                        prevNextPosts: results[1],
                        blog: results[0],
                        comments: results[2],
                        utils: utils
                    });
            }
            else {
                res.redirect('/error?message=5');
            }
        });
    });
}

function setupGet(app) {
    app.get('/blog', function(req, res) {
        processGetBlog(req, res);
    });

    app.get('/blog/:id', function(req, res) {
        processGetBlog(req, res);
    });
}

concurrent processes
From Cyberspace to Main Street, managing multiple, concurrent processes can be challenging!

Keywords: Node.js, node, Express, async, concurrency, callbacks, deeply nested code, async, async.parallel, async.series

Reader Comments

Comment on this Blog Post

Recent Posts

TitleDate
EBTCalc (Android) Version 1.53 is now availableMay 19, 2024
Vault 3 Security EnhancementsOctober 24, 2023
Vault 3 is now available for Apple OSX M2 Mac Computers!September 18, 2023
Vault (for Desktop) Version 0.77 ReleasedMarch 26, 2023
EBTCalc (Android) Version 1.44 is now availableOctober 12, 2021
Vault (Desktop) Version 0.72 ReleasedOctober 6, 2021
EBT Compass is Now Available for Android DevicesJune 2, 2021