Sports Data Sources and Data

Extraction

Gavin Zhang
MIS580

University of Arizona
02-06-2008
Outline
Outline
• Sports Data Sources
– Baseball
– Basketball
– Football
– Olympics
– Greyhound
• Data Extraction
– Case Study: AZGreyhound System
2
Baseball Data
Baseball Data Source
Source

Download the database

• http://www.baseball1.com/ 3
 Data Download
Data Download

• This database contains pitching,

hitting, and fielding statistics for
Major League Baseball from 1871
through 2007.
– The data are provided in Microsoft
Access, CVS and other formats.
– The newest version is Version 5.5.

• The database can be downloaded at:

http://baseball1.com/content/view/57/82/

4
 Database
Database AwardPlayers.csv

playerID awardID yearID lgID

bondto01 Triple Crown 1877 NL
• Detailed description of the database hinespa01 Triple Crown 1878 NL
is available at: heckegu01 Triple Crown 1884 AA
http://baseball1.com/content/view/57/82/ radboch01 Triple Crown 1884 NL
keefeti01 Triple Crown 1888 NL
clarkjo01 Triple Crown 1889 NL
• The database has 21 tables; main duffyhu01 Triple Crown 1894 NL
tables include: rusieam01 Triple Crown 1894 NL
– MASTER Table- Player names, DOB, lajoina01 Triple Crown 1901 AL
and biographical info; youngcy01 Triple Crown 1901 AL
– Batting Table- batting statistics; wadderu01 Triple Crown 1905 AL
– Pitching Table- pitching statistics; mathech01 Triple Crown 1905 NL
– Fielding Table- fielding statistics. mathech01 Triple Crown 1908 NL
cobbty01 Triple Crown 1909 AL
cobbty01 MVP 1911 AL
• Detailed description about each data schulfr01 MVP 1911 NL
field in each table is available. speaktr01 MVP 1912 AL
doylela01 MVP 1912 NL
johnswa01 MVP 1913 AL
…………
5
Basketball Data
Basketball Data Source
Source

Download all of the player and team statistics

• http://databaseBasketball.com/ 6
 Data Download
Data Download

• The website contains the

NBA data from 1947 to 2007
and ABA data from 1968 to
1976 on players, teams,
leagues, all-star games,
awards, and coaches.

• Download at:
http://databasebasketball.com/
stats_download.htm

7
 Database
Database
• This download contains
nine column delimited
files (.txt format), each of
which represents a table
in the database.
• If you open the files up in
excel, you may need to
select Data -> Text to
Columns, then use the
bar ("|") character as the
delimiter.
Teams.txt
team|location|name|leag
ANA|Anaheim|Amigos|A
AND|Anderson|Duffey Packers|N
ATL|Atlanta|Hawks|N
BA1|Baltimore|Bullets|N
BAL|Baltimore|Bullets|N
BOS|Boston|Celtics|N
BUF|Buffalo|Braves|N
CAP|Capital|Bullets|N
CAR|Carolina|Cougars|A
CH1|Chicago|Stags|N
CH2|Chicago|Zephyrs|N
CHA|Charlotte|Hornets|N
CHI|Chicago|Bulls|N
………… 8
Football Data
Football Data Source
Source

• http://www.pro-football-reference.com/ 9
Data Download
Data Download
• A copy of data set (in CVS format) can be downloaded from:
http://ai.arizona.edu/hchen/chencourse/SportsData/Pro-
football-refernce_CSV.zip

• This version contains the game data from 1995 to 2006. The
dataset contains 64,327 players and the games they played in.

• Tables include:
– Master—information about players
– Seasons—the statistics of the players’ records by season
– Games—the statistics of the players’ records by game

• Detailed description about each data field in each table is

available.

10
Database
Database Master.csv
ID la s t na me firs t na me pos ition birth ye a r de but ye a r
AbduKa00 Abdul-Jabbar Karim rb 1974 1996
AbduRa00 Abdullah Rabih rb 1975 1999
AberWa00 Abercrombie Walter rb 1959 1982
AbraDa00 Abramowicz Danny wr 1945 1967
AdamBo00 Adams Bob te 1946 1969
AdamCh00 Adams Charlie wr 1979 2003
AdamCu00 Adams Curtis rb 1962 1985
AdamGe00 Adams George rb 1962 1985
AdamGr00 Adams Grant wr 2000 2005
AdamJo00 Adams John rb 1937 1959
AdamMi00 Adams Michael wr 1974 1997
AdamMi01 Adamle Mike rb 1949 1971
AdamTo00 Adams Tony qb 1950 1975
AdamTo01 Adams Tom wr 1940 1962
AdamTo02 Adamle Tony rb 1924 1950
AdamWi00 Adams Willie wr 1956 1979
AddaJo00 Addai Joseph rb 1983 2006
AdkiJa00 Adkisson James te 1980 2005
AdkiMa00 Adkins Margene wr 1947 1970
AdkiSa00 Adkins Sam qb 1955 1977
…………
11
Some Other
Some Other Football
Football Data
Data Sources
Sources
• http://www.databasefootball.com/
– The website contains the National Football League (NFL) data
from 1922 to 2005 and Australian Football League (AFL) data from
1960 to 1969 on players, teams, leagues, awards, and coaches.

– Data set can not be downloaded directly. The data need to be

extracted from the HTML Web pages by using parsing programs.

• http://www.jt-sw.com/football/
– The website contains the player/coach statistics of NFL from 1920
to present and statistics of AFL from 1960 to 1969.

– Data set can not be downloaded directly. The data need to be

extracted from the HTML Web pages by using parsing programs.

12
Olympics Data
Olympics Data Source
Source

• http://www.databaseolympics.com/ 13
 Data Format
Data Format

• DatabaseOlympics.com is your
source for every Summer and
Winter Olympics medal winner.
– Summer Olympics from 1896-
2004;
– Winter Olympics 1924 -2002

• You'll find every medal winner

for every country with easy links
to each Olympics, sports, and
athletes.

14
Data Format
Data Format

15
Greyhound
Greyhound

• http://66.236.122.233:8080/tracklink/ 16
 Data Format
Data Format

• Data includes daily race programs (videos) and odds charts (.txt file format) for
all US Greyhound tracks.

• Some tracks had both Afternoon and Evening programs. 17

Data Format
Data Format Chart.txt

1st Grade: B Distance: 550 Condition: Fast

DOG WT P O 1/8 Str Fin Time Odds Comment

PTL Jane 63.5 6 3 1 1 1 ns 32.00 11.60 Held At Wire Inside
Silver Speck 68.5 1 1 2 2 2 ns 32.01 2.80 Cutff 1st, Stayd Cls
Jain't It Doug 75 7 7 6 6 3 1.5 32.10 7.50 Closed For Show Outs
Flyer Whitesocks 75.5 8 8 7 3 4 1.5 32.11 2.30 In The Hunt
Flying Detroit 69 5 5 4 4 5 2 32.15 9.00 Not Far Behind Mdtrk
VP Twix Twizala 59.5 3 4 3 5 6 4.5 32.31 4.20 Losing Position Ins
Sergio 73 4 6 5 7 7 5 32.34 13.30 Blocked 1st Turn
Heartattack Jack 71.5 2 2 8 8 8 5.5 32.39 7.10 Bumped 1st Turn
…………
18
 Case Study:
AZGreyhound System
By Rob Schumaker
AZGreyhound System
AZGreyhound System Design
Design
Greyhound Data AZGreyhound

Odds Data Model Building

DB Training / Testing

Race Data Prediction

Metrics Traditional Straight Bets Box Bets

Betting Engine
Accuracy Win Exacta Quiniela

Payout Place Trifecta Trifecta

Efficiency Show Superfecta Superfecta

• AZGreyhound System
20
Greyhound Data
Greyhound Data Extraction
Extraction
• Grayhound data was gathered from
www.trackinfo.com. The Web site links to:
– GreyMatter http://66.236.122.233:8080/tracklink/
– TrackInfo http://www.trackinfo.com/index2.html

• The race and odds data was parsed into a

SQL Server database; then the data was
sent to the AZGreyhound system for
prediction.
21
Example code
public void RacePrograms() throws Exception {
... ...
String URL1 = "http://www.trackinfo.com/trakdocs/hound/";
String URL2 = "/Rpages"; This method picks up the
... ...
OpenConnection2(); overall race information
try { ... ... Data parsing URL
TrackAbbrev = rSet.getString("TrackAbbrev");
and puts it in the database
String URL = URL1 + TrackAbbrev + URL2;
Feed = web.Scraper(URL, 1);
... ...
NumItems = web.NumItems(Feed, "~icons/html.gif");
for(int y = 1; y <= NumItems; y++) {
Feed = Feed.substring(Feed.indexOf("~icons/html.gif"));
Parsing out each
FileName = web.ExtractText(Feed, "<A HREF=\"", "\">"); data field
Feed = Feed.substring(Feed.indexOf("<A HREF="));
FileDate = web.ExtractText(Feed, "NOWRAP>", "</TD>");
FileContents = web.Scraper(URL + "/" + FileName, 1);
FileContents = FileContents.replaceAll("'", "-");

db.Insert2DBProgram(FileName, FileDate, FileContents); }

}
CloseConnection2(); Insert into DB
}
catch(SQLException e) {
System.out.println(e); }
} 22
You can use the sports data sources
introduced in this set of slides for your data
mining project.
You are strongly encouraged to identify other
interesting public sports data sets for your
project.

Thanks!

23