Sales, Abipiel Regina I.

IE 003 – IE41S1 – Operations Research 1

HONOR PLEDGE
“I affirm that I have not given or received any unauthorized help on this assignment and that
this work is my own.”

_________________
Abipiel Regina I. Sales
NAME AND SIGNATURE OF STUDENT

Cite a company that uses or applies the forecasting model to manage their business. Your

discussion must include but is not limited to the following:

1. The nature of business of the company, business problem related to the forecasting model.

2. Application of the forecasting model.

3. Benefit of applying the model.

Blue Apron

Blue apron is a company that provides a service where they send you ingredients and

instructions to cook meal that is planned by a chef for you at home [ CITATION Whi16 \l 13321 ] . It

is what you would call a “Meal Kit Serivice” where you choose their meals for the week, and

you get all the ingredients needed for that meal delivered in a box on your doorstep. Blue apron

offers two plans available for their customers: a 2-person and a 4-person subscription, both with

3 meals per week[ CITATION Pin15 \l 13321 ].

What distinguishes the services of Blue Apron from other physical product businesses is

the perishable nature of their products. Since the ingredients will spoil, they considered the

seasonal availability of the products, and worked to offer innovative packaging, quick transport,

and stable delivery times in order to maximize freshness and avoid product spoilage. From

inventory planning purposes, they also wanted to accurately predict how much produce

customers will buy in the future. That allows Blue Apron to create estimates of future revenue,

waste less inventory, and optimize staffing costs by knowing earlier how many personnel will be

needed to ship merchandise.

Blue Apron applied the forecasting model by compiling data of purchasing and activity

logs that gave them detailed information about their customer’s taste. Blue Apron’s team of data

engineers used these data to create models of customer behavior that can be leveraged to predict

demand, identify how customer preferences change over time, and determine how such changes

are influenced by the recipes that are offered.

One of the core components of their demand forecasting framework is recipe and order

level models[CITATION For17 \l 13321 ].

Blue Apron presents customers each week with a fixed menu of recipes for purchase.

Once the recipes are determined, it is important to accurately forecast how many of each recipe

will be ordered in future weeks, and how many boxes in total will need to be packed. They also

want to understand what is related to these predictions to enhance their ability to make future

predictions.
Sales, Abipiel Regina I.
IE 003 – IE41S1 – Operations Research 1
Module 4 Assignment – Midterms

This problem can be addressed through either an aggregate level or an individual level

strategy. At the aggregate level, the total volume of recipes that is to be predicted can be seen as

a time-varying signal. The analyses are then carried out through time series models, which

typically decompose the original signal into its independent hidden components, such as trend,

slope and seasonality. On the other hand, individual level models aim to capture the behavior of

each individual customer and can also be utilized to deliver personalized content and product

recommendations. However, longer term effects (such as trend and slope) are typically harder to

capture with individual-level models. At the individual level, future purchasing trends are

intimately connected to future subscriptions, cancellations and customer engagement with the

product. Each one of these factors will need to be predicted by appropriate growth and churn

models and through careful feature engineering.

They tried to formalize the problem of customer-level choices from a general standpoint,

without specifically tailoring it to Blue Apron’s product offerings. Assuming that customers are

allowed to purchase 0, 1, or more up to M recipes of the same type at a given time (e.g., 2 recipes

of type beef, 1 recipe of type poultry, etc.), it is also assumed that each order can include 0, 1, or

more up to N recipes. An order containing 0 recipes will signify a customer who does not wish to

order.

Given the history of user purchasing behavior (user features), types of recipes offered

(recipe features) and other factors such as seasonalities and holidays, Blue Apron wanted to

predict the menu configuration that customers will choose on a given future week. The process
Sales, Abipiel Regina I.
IE 003 – IE41S1 – Operations Research 1
Module 4 Assignment – Midterms

of mapping from input features to this multivariate multinomial output is depicted in the diagram

below.

Multivariate multinomial recipe model

Generally speaking, inferring whether a customer will choose recipe k a

number of times j involves computing the conditional probability

where the index l indicates any recipe other than k, and X denotes the

set of input features. The right-hand side of the equation above is often

referred to as softmax function and is used to represent a categorical

distribution, such as the multinomial distribution. When linear classifiers are

used, the functions z are linear combination of the input features X, that is:

Training involves finding the values of the alternative- and choice-specific intercepts, α,

the feature coefficients, β, and the association parameters, φ, such that the difference between the

predicted and actual outcome Y is minimized. The association parameters are used to quantify

the effect of other recipes upon the choice of recipe k whereas the feature parameters will capture

the effect of the input features on the observed outcome. When non-linear classifiers such as

artificial neural networks are used, the functions z are implicitly learned by the classifier and can

become arbitrarily complex. Because the input features can no longer linked to the learned

coefficients directly the prediction becomes harder to interpret. This lack of interpretability can

be detrimental within the e-commerce domain where various stakeholders need to understand the

logic behind the inferences made by the algorithm. An interpretable model allows for

adjustments like new marketing and inventory planning strategies, as well as improvements to

product offerings based on customers’ preferences and habits.

From a practical standpoint, interpretability is not the only desirable property for the

recipe-level model. Training the classifier presented above requires implementing log-likelihood

functions involving nested summations over all customers, recipe types and number of recipes

than can be purchased for each recipe type. If the product offerings change, e.g. because one

recipes type is added to the weekly menus, the log-likelihood functions will also need to change.
Sales, Abipiel Regina I.
IE 003 – IE41S1 – Operations Research 1
Module 4 Assignment – Midterms

As a result, the old data can no longer be used for training the new model and the prediction

across all outcomes will remain inaccurate until enough new data becomes available.

In order to overcome these difficulties, Blue Apron investigated the use of linear

classifiers and ensemble-based classifiers that are trained to model each of the N outcomes

separately. The assumptions they made is that the choice about one recipe type is independent of

the choices over all other recipe types, as long as the input features contain information about all

recipes that the customers can choose from. This effectively translates to assuming that the

association parameters can all be factored into the recipe features. As a result, the introduction of

new recipe types will not affect the accuracy of all other recipe models, as these latter can still

use the old data for training.

The model presented earlier allows forecasting volumes at the recipe level. For the

purpose of easing inventory and demand planning tasks, it is equally important to be able to

predict the total number of recipes that will go into each order, no matter the recipe type. The

total recipe volume can either be inferred from the recipe-level models, or it can be predicted

using another model: an order-level classifier. Such a classifier should be used to capture

restrictions that the business may impose on the total number of recipes that make up an order

(e.g. from a business perspective may be more beneficial not to sell orders with less than 2

recipes or more than 5 recipes). Furthermore, the order-level model may be suitable to capture

recipe-independent behaviors and habits, such as the tendency of customers to order the same

number of recipes every week, regardless of the meals being offered. The order-level classifier
Sales, Abipiel Regina I.
IE 003 – IE41S1 – Operations Research 1
Module 4 Assignment – Midterms

can share some or all of the features used by the recipe models, but it will be trained to predict

the total number of recipes that the customers are likely to purchase.

The features Blue Apron typically use fall into three main categories:

 Customer-related features

 Recipe-related features

 Seasonality features

Customer-related features encode the frequency at which customers are likely to order in

a given week, regardless of what is on the menu on that week: these features are

particularly useful because they allow learning whether a given user is one who orders

most weeks, one who orders very infrequently, or one who has likely stopped ordering

entirely. Their recipe-related features encode whether a specific recipe has certain

attributes (for instance, whether it has an ingredient such as salmon or asparagus, or a

cuisine like Japanese), how many other recipes on offer that week contain the same

attributes, and how likely users are to have ordered recipes with those attributes before.

Together, these three categories of recipe-related features allow sussing out users-recipe

affinities from the past, and inferring which upcoming recipes users are likely to order in

the future. Finally, seasonality features can be used to model periodic purchasing

patterns, thus allowing discerning when order rates may be higher or lower during the

year. In their most simple form, seasonalities can be encoded by a single one-hot feature

vector, containing as many elements as the number of weeks (for weekly seasonalities) or
Sales, Abipiel Regina I.
IE 003 – IE41S1 – Operations Research 1
Module 4 Assignment – Midterms

months (for monthly seasonalities) in a year. This vector will be 0 everywhere, except for

one single 1 representing the week or month to be predicted.

Through regression analysis, Blue Apron’s engineering team has been able to measure

the precision of its forecasting models. The team reports that, overall, the root-mean-

square error of their projection of future orders is consistently less than six percent,

indicating a high level of forecasting accuracy.

By employing predictive analytics to better understand customers, Blue Apron has been

able to improve its user experience, identify how subscriber tastes change over time, and

recognize how shifting preferences are impacted by recipe offerings [ CITATION Gav19 \l

13321 ].

References
1. White, Dana. How Does Blue Apron Work - Details About the Process of Ordering and Cooking Blue
Apron Meals. a Slob comes Clean. [Online] September 7, 2016. [Cited: April 16, 2021.]
https://www.aslobcomesclean.com/2016/09/how-does-blue-apron-work-details-about-the-process-of-
ordering-and-cooking-blue-apron-meals/.

2. Pinola, Melanie. The Best Meal Kit Services: Blue Apron vs. Hello Fresh vs. Plated. lifehacker. [Online]
January 29, 2015. [Cited: April 16, 2021.] https://lifehacker.com/the-best-meal-kit-services-blue-apron-
vs-hello-fresh-1682517921.

3. Nantes, Alfredo. Forecasting Demand at Blue Apron. Blue Apron. [Online] August 21, 2017. [Cited:
April 17, 2021.] https://blog.blueapron.io/forecasting-demand-at-blue-apron-ba62d6af5da2.

4. Gavin, Matt. 3 EXAMPLES OF BUSINESS ANALYTICS IN ACTION. Harvard Business School→HBS Online.
[Online] Harvard Business School Online, January 15, 2019. [Cited: April 16, 2021.]
https://online.hbs.edu/blog/post/business-analytics-examples.
Sales, Abipiel Regina I.
IE 003 – IE41S1 – Operations Research 1
Module 4 Assignment – Midterms