Elkhorn Guidelines

Date Updated: 10/14/2019

Overview

A document is a generic term referring to any of multiple sources we pull content

from to be categorized. Your task is to understand the purpose of each document
and to match it with the best-fit subcategory. Example documents can be Facebook
ads, Internet webpages, Facebook Marketplace posting, and many others.

Categories are organized into a taxonomy, which is a hierarchical structure of terms

that starts from broad categories and narrows down to more specific sub-
categories. Our taxonomy is comprised of three purposes: Product, Service, and
Content.

Throughout each of the workflows, keep in mind the following labeling principles:
1. Categorize to the most specific, while still inclusive, subcategory. For
example, if a laptop is presented in a document as a product, categorize to
Product > Electronics > Computers > Laptop, and not just Product >
Electronics > Computers. While Computers is an appropriate category to
describe what laptops are, there is a subcategory of ‘Laptop’ that is more
specific.
2. If you are unfamiliar with the product/service/content featured in a
document, feel free to perform a brief side search using your favorite search
engine. Please do not exceed 60 seconds for side searching.
3. For cases when you are unsure, refer to the following:
a. Bias towards identifying the purpose as Product over Service, and
Service over Content.
b. Bias towards the image over accompanying text.

How To Categorize A Document

You will be presented with a single page and will be asked to select the most
specific, but still inclusive, label. The document will be presented on the left side
along with any additional information about it. On the right side will be the
interactive section for you to complete the task and submit.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 Labeling Instructions

Step 1 - Determine if there are any document issues that prevent you from
categorizing the document.
Issue Description Instructions
Page Load Document fails to load any Select "Page Won't Load".
Error information. SRT widget problem
Missing The document does not provide Select "Missing Context".
Context enough contexts for you to
decide a category. For example, a
login page or a blacked out
screen.

If you do not understand the

content, you are allowed to side
search to gain the necessary
understanding; however, this is
not a scenario of Missing
Context.
Wrong If some or the entire document is Select "Wrong Language".
Language in a language you do not
understand, preventing you from
understanding its purpose.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 Sensitive If a document contains or
potentially contains any of the
following:
- Child Exploitation and/or Child
Nudity
- Self Injury and Suicidal Content
- Credible Threats, Violence or
Calls to Violence
- Sexual Content and/or Nudity
- Hate Speech
- Acts of Terrorism
- Human Trafficking
- Bullying and Harassment
Select "Sensitive".
Escalate to your manager
immediately and report the Job
ID if the document contains:
1. Child Exploitation and/or
Child Nudity
2. Self-Injury/Suicidal
3. Time-Sensitive Credible
Threat

If any of the scenarios apply, select the correct button at the bottom of the page. The
job automatically moves to the next job if selected. If no issues, skip to step 2.

Step 2: Choose whether the core purpose of the document is a Product, a

Service, or Content.

Documents may reference multiple purposes, but the goal is to consider which of
those is intended to be most important to the user. Consider all available
information on the tool to aid in selecting a purpose.

Familiarize yourself with the definitions in the table below and choose the most
appropriate document purpose.
Product Service Content
A specific, physical A service is an Content is an idea or
Definition good that a person can intangible thing that information that is
use and a person experiences shared without
retain. Products are in real time, but commercial intent.
items that can be cannot retain, and All media (music,
bought/sold and may that can start or stop movies, books,
be offered as a free trial being provided. games, etc.) are
or as a sample. considered Content.
Clothing, Shoes & Beauty treatments, People profiles,
Examples Jewelry, and Furniture legal consultation, media (e.g., music,
are examples of items and food served in a movies, books,
that can be physically restaurant are all newspapers,
bought or sold. examples of services magazines), and food
you experience. recipes are examples
of content.
Note:
• Service — We consider businesses, software & applications, and live
performances under the Service purpose, all of which provide a service

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 function to people even if they require a physical item in order to access. For
example, a ticket to access a live performance or a CD to access the software
& application.
• Content — Content contains all topics that are not specifically captured
under the Product and Services purpose, including people. In general, the
Content purpose is the catchall purpose for when the document is not
promoting/representing a specific Product or Service.
Special Considerations:
• Food: When the document presents food that is acquired from a specific
restaurant, consider it as a Service (contrary to the usual bias of Product >
Service).
o Example 1: A document promoting a specialty pizza from a pizza shop
should fall under Service > Food, rather than Product > Food,
Beverages & Tobacco.
o Example 2: A document from a catering company promoting menu
dishes should be part of Services as well.
o Example 3: A document promoting a food delivery service, should fall
under Service > Food. This includes subscription services such as
HelloFresh, UberEats, GrubHub, etc.
• Digital Media: Digital goods are to be categorized under the Content
purpose. If both a digital good (e.g., mobile app games, movies, music, video
games) and the service that provides the digital goods (e.g., Netflix, Spotify,
YouTube) are presented in the document, you should choose to categorize
the digital good over the service. For example, if Netflix is promoting a
specific movie, categorize the movie under content and not to Netflix as a
service.
o Physical versions of these digital goods (e.g., a book, dvd, physical
copy of a video game) still fall under the Content purpose as the
specific format is not important.
o Mobile apps, excluding mobile app games, are considered Service.
o Exception: If a document is promoting a digital media at a specific
physical venue (e.g., movie at a movie theater), then you should
categorize to the venue/service under the Service purpose. If the
document is non-specific about the physical location, categorize the
document under Content purpose. For example, an advertisement for
a movie playing at a local theatre should be categorized to Service
while an advertisement for a movie playing nationwide should be
categorized to Content.
• Events: For events, we want to identify why people would be interested in
the event and categorize the main service or product presented at the event.
o Example 1: A document selling tickets to a live performance at a
venue would be categorized under Service > Arts & Entertainment
because the consumable part of the event is the live performance.
o Example 2: A document promoting an event with a famous Home &
Garden TV personality focused on real estate investing, the consumer

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 part of the event isn’t the host but rather is the real estate investing
topic. Therefore, we want to categorize the real estate investment
as 'Services > Finance > Investing > Real Estate Investment'.
o Example 3: A document is promoting a boating trade show event
where people can buy and sell boat parts. The reason people would be
interested in this event would be because they are interested in
buying or selling boat parts. Therefore, since the valuable consumer
part of the event is around boat products, the correct category
is 'Products > Vehicles & Parts > Vehicle Parts & Accessories >
Watercraft Parts & Accessories'.
o Note: The 'Services > Arts & Entertainment > Events > Ticket
Distribution & Resell Service' category is reserved for businesses that
distribute tickets like TicketMaster and StubHub, but not when a
specific event is promoted, even if they are the ticket provider.
• Contests or Sweepstakes: Determine the purpose of a contest or
sweepstakes by categorizing the top prize or promotion.
o Example 1: The document invites you to participate in a contest for
the chance to win a necklace, choose 'Products > Apparel & Shoes &
Jewelry > Jewelry > Necklaces'.
o If the prize is money, categorize to Services > Arts & Entertainment >
Gaming & Play > Real Money Gaming.
• Quizzes & Questionnaires. Categorize quizzes and questionnaires to the
best-fitting topic under the Content Branch.
o Example 1: A Buzzfeed quiz about movies should be categorized as
'Content > Movies, Television & Web > Movie Genres'.
Flowchart
Use as a general guide to think about how to categorize the document into the correct
branch. This may not be comprehensive to cover all considerations in every document
type.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 Step 3: Continue categorizing the document to the most specific, but still
inclusive, subcategory.
After choosing the purpose, additional sub-categories will appear. You should keep
choosing the most appropriate subcategory for the document until you reach the
lowest level of granularity possible that represents the product, service, or content.
• When to select Multiple Items: In cases when two or more sub-categories
apply or if the higher-level category is the best-fit, choose multiple items.
o Example 1: A document is offering a variety of baby clothing; onesies,
dresses and matching sets. Label as Products > Apparel, Shoes &
Jewelry > Apparel > Clothing & Clothing Accessories > Baby &
Toddler. Clothing > Multiple Items because there are multiple sub-
categories of clothing applicable for each item within the 'Baby &
Toddler Clothing' subcategory.
o Example 2: A document selling a bedding set consisting of a quilt,
pillow cases, and bed sheets would be labeled as 'Products > Home,
Lawn & Garden > Linens & Bedding > Bedding > Multiple Items'
because there are multiple equally dominant products that fit within
the 'Bedding' sub-categories.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 o Example 3: A document featuring a performing art school offers
classes in both dance and theater. Label as 'Services > Education >
Professional Schools & Certification > Art Schools > Multiple Items'
because “dance academy' and 'drama & theater' are both applicable
sub-categories under 'Art Schools'.
• When not to select Multiple Items: The document may feature multiple
sub-categories of products, services, or content, but if one subcategory is
clearly more dominant, then categorize only the dominant product, service,
or content. For example, an ad for a mobile phone where the phone is the
main focus but also mentions the phone charger and phone case, consider the
mobile phone as the dominant product to be categorized because the main
offering is the phone while the phone case is supplemental to the phone.
• When to select None of the Above: If you feel that none of the sub-
categories are a good fit and the subcategory that describes the product,
service, or content is missing, choose “None Of The Above”.
o Example 1: An ad features a musician who specializes in musical
accompaniment for live events. Label to Services > Arts &
Entertainment > Music & Nightlife > None Of The Above because there
is not a subcategory for “musical accompaniment” under the 'Music &
Nightlife' subcategory.
o Example 2: A group post features historical photography of small
towns. Label to 'Content > Arts, Crafts & Culture > Arts > Photography
> None Of The Above' because 'historical photography' is not an
available subcategory of 'Photography'.
o Example 3: A document features a horse pulling business. There is
not a subcategory available for “horse-pulling” so categorize as far as
you can, to 'Services > Sports & Recreation > Horse Equestrian > None
Of The Above’.
o If there are no applicable sub-categories beyond the core purpose
(Products, Services, or Content), select the core purpose and then
'None Of The Above’.

FB Events (HRT to review below)

What is Facebook Events?

Facebook Events build meaningful connection by letting users organize and respond
to gatherings in the real world with people on Facebook.

Document Components

Consider each of the document components below for helpful clues in determining
the event's purpose.
CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019
 1. Event Name: the name of the event.
2. Event Keywords: keywords selected by the event host. Note: most events will
not have keywords.
3. Event Description: details about the event, what it is for, what to expect,
logistics.
4. Cover Photo: provides an image of the event or provides clues to the
theme/occasion.

Labeling Guidance
1. Determine the purpose of the event. As you evaluate the document, ask
yourself: “What is the main purpose for this event? What is the main
interest that brings people together?
2. For events where the main attraction is a person, categorize what the
person is most known for to the Content branch. If you need more
information to determine how to categorize the person feel free to conduct a
quick side-search using your search engine of choice.
3. Most events will be categorized under the Services purpose as there is
typically a service or real-world experience offered.
4. Categorize ticketed events to the most relevant service or product purpose
the event is about and not to the 'Services > Arts & Entertainment > Events >
Ticket Distribution & Resell Service' subcategory. This category is reserved
for businesses that distribute tickets like TicketMaster and Stub Hub, but not
when a specific event is promoted, even if they are the ticket provider.
5. Although ‘Services > Arts & Entertainment > Events’ may be applicable to all
events, do not categorize to this category unless the event can be categorized
to its sub-categories. Do not stop at ‘Services > Arts & Entertainment >
Events’.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 6. Refer to a person’s Wikipedia bio and choosing the first option that describes
the person. For example, Tom’s Hanks Wikipedia bio describes him as ‘an
American actor and filmmaker’. “Actor" is listed first before "filmmaker" so
we will categorize Tom Hanks to ‘Content > Movies, Television & Web >
Movie & TV Industry > Acting & Actors’.
7. For personal events, such as birthdays, weddings, and categorize to ‘Content
> Arts, Crafts & Culture > Culture > Holidays & Celebrations > Personal
Celebrations’.
8. Retail events that promote a specific store or shopping experience can be
categorized to the business under ‘Services > Retail’ > best-fit subcategory.
9. Live performance events can be categorized to ‘Services > Arts &
Entertainment’ > best-fit subcategory.

Event Categorization Table

Type of Event Categorize to Tip

1 Live Services>Arts & Categorize all live musical
performaces: Entertainment>subcategory performances, including music
dance, festivals, night club/D.J.
theater, perofrmances and concerts to
cinema, Services>Arts &
comedy, Entertainment>Music &
festivals, live Nightlife
music,
nightclubs,
DJs, etc
2 Live sporting Services>Sports> 'Live sports' includes both non-
events subcategory professional and professional
matches.
3 Food and Services>Food & Categorize to the specific type
drink related Drink>subcategory of food/drink subcategory. For
events example, a beer festival is
categorized to Services>Food &
Drink>Alcohol>Beer>Multiple
Choice
4 Restaurant Services>Food & Categorize to the specific
and food Drink>Restaurants & resturaurant or food type. For
events Shops>subcategory example, an event where bbq is
the main attraction categorize
to Services>Food &
Drink>Restaurants &
Shops>Barbeque
5 Retail and Services>Retail>subcategory For retail events that do not
shopping promote a specific product,
related events categorize to the business type.
CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019
 For example, an event for a
garage sale is categorized
to Services>Retail>Used
Merchandise Stores. An event
related to an online market or
auction such as Amzaon or
Ebay is categorized
to Services>Retail>Marketplace
& Auctions.
6 Movies, film Services>Arts & For events that feature a movie
screenings, Entertainment>Movies & or cinematic entertainment as
live cinema Television>Movies>NONE the main attaction, categorize
related events OF THE ABOVE to Services>Arts &
Entertainment>Movies &
Television. This includes
movies viewed in a movie
theater.
7 Religous Services>Civic and For religious themed events
ceremonies & Religious>Religious such as religious study groups
spiritual Services>subcategory and memorials categorize to
observances the best-fit
subcategory Services>Civic and
Religious>Religious Services.
For memorials and funerals
categorize to this node and
stop.
8 Fundraisers, Services>Civic and Please categorize to the best-fit
charity Religious>Charity & cause type
related Philanthropy under Services>Civic and
events, Causes>subcategory Religious>Charity &
philanthropic Philanthropy Causes and not to
causes any other event theme. For
example, a football match to
raise money for charity is
categorized to Services>Civic
and Religious>Charity &
Philanthropy Causes>Relief of
Poverty and not to football.
9 Live All events that feature a
performances, Services>Arts & performing art as the main
including all Entertainment>Performing attraction can be categorized
performing Arts>subcategory to
arts Services>Arts &
Entertainment>Performing
Arts. For example, an event for

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 the musical Hamilton is
categorized to Content>Arts,
Crafts &
Culture>Arts>Theater>Musical
Theater.
10 People, Content>subcategory For events where the main
celebrities, attraction is a person,
personalities categorize to the subcategory
that describes that person or
what the person is most known
for under Content. For
example, an event featuring an
author talk and books signing
is categorized
to Content>Books &
Literature>Authors & Writers.
11 Buying & Products>subcategory For an event where buying or
selling selling a product is the main
focus please categorize to the
Product branch. Please note
that 'Product' is the top-level
category and you will need to
categorize to the subcategory
the describes the product. For
example, a clothing swap event
is categorized
to Products>Apparel, Shoes &
Jewelry>Apparel>Clothing &
Clothing Accessories>Multiple
Items
12 Events Content>News & Please categorize the cause,
related to Politics>subcategory and not the ativitity, to the
activisim or a best-fit subcategory under
political Content. Many political
cause. subjects are located under the
News & Politics subcategory
branch. For example, a strike
for fair wages can be
categorized to Content>News &
Politics>Workforce & Labor. A
protest to bring awareness to
climate change can be
categorized to Content>News &
Politics>The
Environment>Environmental

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 Activism.

Event Type Example

Categories
1 URL Musical https://z-m- https://z-m- https://z-m-
performances www.facebo www.faceboo www.faceboo
/concerts/ ok.com/even k.com/events k.com/events
festivals ts/66269663 /2158388559 /236098710
0772099 27009 603569
2 Musical Band
performances performances,
= Services>Arts concerts &
& music
Entertainment> festivals,
Music & nightclub
Nightlife events, DJs
3 Purpose>
Subcategory
4 URL Live Sports! https://z-m- https://z-m- https://z-m-
www.facebo www.faceboo www.faceboo
ok.com/even k.com/events k.com/events
ts/78740856 /8058807564 /805880756
4973794 71668 471668
5 Live Sport Professional &
Matches= local games
Services>Sports
& Recreation
6 Purpose> Services> Services>Arts Services>Arts
Subcategory Arts & & &
Entertainme Entertainmen Entertainmen
nt>Live t>Live t>Live
Sports> Sports>Car Sports>NONE
Hockey Races OF THE
ABOVE
7 URL Personal https://z-m- https://z-m- https://z-m-
Events www.facebo www.faceboo www.faceboo
ok.com/even k.com/events k.com/events
ts/17114868 /8181560019 /241101656
0426418 50662 568119
8 Personal Events Weddings,
engagements,
birthdays,
graduations,
babyshowers,
memorials,

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 anniversaries
9 Purpose>
Subcategory
1 URL https://z-m- https://z-m- https://z-m-
0 www.facebo www.faceboo www.faceboo
ok.com/even k.com/events k.com/events
ts/64905255 /6621049675 /162563654
5551102 80508 693492
1 Fundraisers & Medical cures
1 Causes & assistance,
1 Purpose> Services> Services>Civic Services>
2 Subcategory Civic and and Health &
Religious> Religious> Medical>
Charity & Charity & Women's
Philanthropy Philanthropy Health>
Causes> Causes> NONE OF
Medical Multiple Items THE ABOVE
Assistance

Confusing Cases

Case #1: Event has more than one purpose

Solution: For events that have more than one purpose, identify the multiple
purposes and ask yourself “which purpose best describes the common interest of the
event for most people in attendance?” Choose the best-fit purpose that would be of
interest to the majority people attending the event.

Example 1: A charity event fundraising for a cause. Charity fundraiser for a cause
has two purposes: the action of fundraising and the cause that people are
fundraising for. Although fundraising is an important purpose of the event, the
cause is why people would be likely interested in the event in the first place.
Categorize the cause and not action.

Example 2: A Star Wars themed fun run. People may be interested in Star Wars but
may also interested in running the 5k.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 If in doubt and the purposes are equally dominant, bias towards categorizing
Product > Service, and Service > Content.

Case #2: The information provided in the event information is sparse and the
event looks to have multiple equally dominant purposes.
Solution: If the purposes are equally dominant, find a category that captures the
majority of the purposes.

Example 2: Glendorafest 2018 - A festival with food, drinks, and live music as
inferred from the cover photo with the tagline “Beer, Brats, & a Band”. The tagline
doesn’t allude to one purpose being more important than the others. The correct

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 category is ‘Services > Food & Drink > Food Festivals’ as it captures 2 out of 3
purposes, food and drinks.

Case #3: Event is in memorial, honor or in celebration of something

Solution: Identify the common purpose why people would want to attend the event
and categorize that purpose.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 Example: The Memorial Murph is a communal “workout and celebration of
community in honor of veterans and fallen heroes”. Although the event title
mentions celebration of community, the reason why people would come is to
partake in the cross-fit workout.

The correct category is ‘Services > Sports & Recreation > Fitness Workouts >
CrossFit’.

Case #4: A non-performance event with a personality (e.g., an author’s book

signing tour)
Solution: For purposes centered on individuals, categorize to the content purpose.

Example: An in-person event with an author of a new book.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 The correct category is ‘Content>Books & Literature>Authors & Writers’.

Case #5: A retail business promoting a day/occasion

Solution: With events centered around a promoted product, categorize the product.

Example: A comic Books shop promoting free comic book day.

The correct category is ‘Content > Books & Literature > Books > Book Genres >
Comics & Graphic Novels’.

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019

 Case #6: Themed events
Solution: We want to Bias towards the action/experience rather than the theme.

Example: Star Wars themed fun run, bias towards the fun run.
https://z-m-www.facebook.com/events/245041443100275

CONFIDENTIAL TO APPEN-DO NOT FORWARD Last Updated 10/14/2019