We’re happy to announce a new WordPress Cron infrastructure for our platform. In this post, we’ll take you through why we did this, how we did it, and what problems it solves for our WordPress VIP customers.

The WordPress VIP platform provides performance, speed, and scale to the highest traffic sites. Each component and service we support plays a role in that mission. The new VIP Cron infrastructure ensures your site can schedule one-off tasks, offload intensive processing, and run repeated actions reliably, on time, and without additional developer effort.

Our Cron implementation builds on the core WordPress Cron API, for maximum code portability from the WordPress ecosystem and familiarity for your engineering teams.

High traffic sites and WordPress Cron

The WordPress Cron system allows scheduling of asynchronous events, such as publishing a post at a future date or sending out a survey a few days after completing an order. It also facilitates running repeated tasks, such as syndicating content between sites or ingesting videos from third party video services. The core WordPress Cron system works well for many WordPress sites every day.

Traditionally, WordPress Cron is triggered by normal traffic to your WordPress site. Regular visitors trigger an AJAX request back to the server that identifies and runs pending tasks. This approach works great for many sites, as it has no additional dependencies or setup requirements. However, ease-of-use comes with a few trade-offs:

• Unreliable triggers — cron is only triggered when there is traffic to your site
• Shared resources — the jobs run on the same server as regular web requests, so intensive cron jobs can negatively affect site performance
• Hard to scale — difficult to process many jobs in parallel, or handle very large numbers of scheduled events

VIP sites rely on Cron for mission-critical functionality that must work reliably every time. Our new Cron infrastructure is designed to ensure the reliability and scalability of cron events on every VIP site.

Smarts, brawn, and confidence — pick all three

We’ve improved three main areas of Cron for our customers:

Smarter process control. By default, WordPress Cron processes events serially. This is fine for sparse queues composed of light tasks but enterprise sites often require offloading a long running task to Cron for asynchronous processing. These events function like slow moving traffic on a single lane highway. Subsequent events can be processed late due to being “stuck behind” a slow moving task.

An enterprise WordPress Cron needs to be able to process offloaded tasks efficiently without impacting the regular operation of the site.

Handling giant queues. A large Cron queue can cause issues where the size of the queue exceeds the capacity of a single option and object caching.

An enterprise hosting platform must handle enterprise-sized queues.

Mission-critical scheduling. Initiating an event in core WordPress Cron relies on unrelated web requests to trigger events. This dependency can cause issues with the event processing regularity and timeliness.

An enterprise WordPress Cron solution must run scheduled events on time, every time.

In short, we wanted to ensure that the Cron infrastructure for each WordPress VIP site was reliable, powerful, and dedicated to that site, just like the rest of the WordPress VIP platform. We wanted resource intensive tasks to be offloaded to dedicated containers, rather than running on the same resources used to serve web requests. And to ensure tasks for one site did not interfere with other tasks, or with the operation of another site.

It was also important that we fully supported the core WordPress Cron API, so our clients can utilize existing plugins and themes without refactoring code or learning a new API.

A better WordPress Cron

Our Cron Control plugin (open source code) builds on the core WordPress Cron system, and is the basis for our Cron enhancements. Cron Control provides a carefully optimized SQL table for WordPress Cron events. This approach satisfies the highly concurrent querying we commonly see on WordPress VIP sites. Each named event in the queue is handled in parallel with other events, allowing greater event handling capacity.

Cron events on a WordPress VIP site run on dedicated containers using an “event runner” written in Golang (open source Golang runner code). Using our container-based infrastructure allows us to scale the number of containers to meet the demands of the particular site, independently of the site’s web traffic

The Cron Control event runner first spawns a batch of “event retrievers” that collect events to be run. For a WordPress multisite, this means spawning parallel event retrievers to collect the events for each individual subsite within the multisite. Once events are all retrieved, they are farmed out to a dedicated pool of “event workers” that execute WP CLI commands to run each event.

Busy sites may have several Cron runners in separate containers all processing the queue simultaneously. Our VIP Cron infrastructure takes particular care to orchestrate the activity of the event workers in the different containers, to avoid clashes with two workers processing the same event.

While the event runner is written in Golang, it interacts closely with WordPress through WP CLI commands provided by the Cron Control plugin. All configuration (such as enabling/disabling cron itself and parallelization limits) is via WordPress hooks in the site code. This makes controlling cron processing easy and familiar for WordPress developers.

Ensuring scheduled posts are published on time is a concern for many clients. Cron Control gives priority to ensuring scheduled post events are run when they are found, and that the list of scheduled posts is up to date.

Good monitoring, smooth operations

The Cron Control system is monitored by a Node.js application, itself hosted on the VIP platform (yes, we host Node apps too!).

The monitor uses a series of dedicated authenticated REST API endpoints on each WordPress VIP site (and each subsite on each WordPress multisite) to ensure that event queues remain within acceptable parameters, that the events within the queue are executed in a timely manner, and that execution is proceeding smoothly.

If any issues are detected, the WordPress VIP team is alerted and investigates the problem.

On time, every time

Our new Cron infrastructure serves the complex and mission-critical needs of some of the most demanding enterprise applications on the web. Contact us to learn how you can benefit from the same peace of mind and let WordPress VIP power your digital experiences.

For existing customers, we have a separate WordPress VIP Lobby post where we take you through the steps to take advantage of our new VIP Cron infrastructure.