The definitive guide (Kristina and Michael)

1. Introduction
MongoDB
- Document-oriented database (Key-Value Pair).
- Wherever Key is reffered (Consider it as column).
- Full-featured data store that is scalable, flexible, and fast.
- Main purpose is to make scale out easier.
- Table = Collecton :: Row = Document.
- Schema free (no predefined number of columns or datatype is needed).
- Schema free features gives developer lot of flexibility to work with evolving
data models.
- Features :- Indexing (Secondary, unique, compound, geospatial and variety of fast querie
s).
- Stored java script (Developers can store javascript functions and values on
the server side).
- Aggregation (Supports mapreduce and other aggregation tools)
- Fixed size collections (Capped collections are fixed in size and are useful
for certain type of data, such as logs).
- File storage (Supports easy to use protocol for storing large files and meta
data).
- Speed :- Uses binary wire protocol to interact with server (instead of protocol with
more overhead like 'HTTP/REST')
- Dynamic padding to documents and preallocate data files to trade extra space
usage for consistent performance
- Uses memory maaped files in the default storage engine, which pushes respons
ibility of memory management to OS
- It has dynamic query optimizer that remember fastest way to perform query.
- Whenever possible, the database server offloads processing and logic to the
client side
- Simple Administration: - Automatic fail over (Slave becomes master)
- Once cluster know that new node exists it automatically integrates and confi
gurte it.
- Server is design to handle configuration automatically (but allows user as w
ell to tweak setting).
================================================================================
===================================================================
2. Getting Started
General
- Document = Row
- Collection = Table
- MongoDB can have multiple databases and each of which can have its own collec
tions and permissions
- Comes with javascript shell, useful for admin and data manipulation
- Every document has a special key, "_id", that is unique across the document s col
lection.
Document
- It's a Key-Value pair.
e.g. : {"foo" : 3, "greeting" : "Hello, world!"}
- Keys in document are always string.
- . and $ are special properites. Read doc carefully.
- MongoDB is type+case sensitive.

e.g.: a) {"foo" : 3} <-> {"foo" : "3"} :: One has integer type value and oth
er has string type value.
b) {"foo" : 3} <-> {"Foo" : 3} :: One has "foo" starting with small f
and other has "Foo" starting with capital F.
Collections
- Collection = Table
- A collection is a group of documents. If a document is the MongoDB analog of
a row in a relational database, then a collection can
be thought of as the analog to a table.
- Schema-Free
-> both of the following documents could be stored in a single collection:
{"greeting" : "Hello, world!"} {"foo" : 5}
- Collection name should be UTF8 string, should not be NULL, should not be sta
rt with system.(a prefix reserved for system collection),
should not contain reserved charact "$".
SubCollection: Subcollections are a great way to organize data in MongoDB, and
their use is highly recommended. (Need to revisit)
Databases: MongoDB groups collections into databases. A single instance of Mong
oDB can host several databases.
- Each of which can be stored on different disk.
- Database name should be UTF8 string, should not be NULL, should not be start
with system.(a prefix reserved for system collection),
should not contain reserved charact "$",".","/","\","\0", should be all lowe
r case, length should be maximum 64bytes.
- Database name will actually end up as files on your file system.
- Reserved DB names (admin, local, config)
- admin: if user added to admin then it will inherits permissions of all dat
abases. shutdown and listing DB can run only from admin DB.
- local: this DB should not be replicated and should be used only for local
collections.
- config: When Mongo is being used in a sharded setup (see Chapter 10), the
config database is used internally to store information about the shards.
NameSpace: By prepending a collection s name with its containing database, you can
get a fully qualified collection name called a namespace
e.g.: For instance, if you are using the blog.posts collection in the cms dat
abase, the namespace of that collection would be cms.blog.posts.
- Namespaces are limited to 121 bytes in length.
Start Mongo Server
- $ ./mongod
- When run with no arguments, mongod will use the default data directory, /dat
a/db/ and port 27017.
- HTTP will listen to port 28017 (http://localhost:28017).
MongoDB Shell
- MongoDB comes with a JavaScript shell that allows interaction with a MongoDB
instance from the command line.
- $ /opt/mongoDB/bin/mongo
- x = 200
- x / 5;
- JavaScript libraries
- > Math.sin(Math.PI / 2);
- > new Date("2010/1/1");
- > "Hello, World!".replace("World", "MongoDB");
- UDF function
- function factorial (n) {

... if (n <= 1) return 1;

... return n * factorial(n - 1);
... }
- > factorial(5);
MongoDB Client
- the real power of the shell lies in the fact that it is also a stand-alone
MongoDB client
- On startup, the shell connects to the test database on a MongoDB server and
assigns this database connection to the global variable db.
- Some add-ons which works on Shell, won't work with java script alone.
e.g.: > use <dbName>
: > db (this will show db name)
Basic operations with Shell (CRUD)
Create:
objStudent = {"RollNo": 1, "Name": "Jon", "Date": new Date()}
db.Student.insert(objStudent);
objStudent = {"RollNo": 2, "Name": "Don", "Date": new Date()}
db.Student.insert(objStudent);
Read(Find):
db.Student.find(); <finds all>
db.Student.findOne(); <finds one>
Update:
var st1 = db.Student.findOne();
st1.Name = "Son"
db.Student.update({RollNo: 1}, st1)
Delete:
db.Student.remove({RollNo: 1});
db.Student.remove({})
Tips for using shell:
- > help
- show dbs, show collections, show users, show profile, db.help(), db.collect
ion.help(), etc.
- Figuring out what a function is doing is to type it without the parentheses
. This will print the JavaScript source code for the function.
Note: For application there are methods provided for each things which needs to
be studied and used.
DataType
- Documents in MongoDB can be thought of as JSON-like
in that they are conceptually
similar to objects in JavaScript.
JSON s expressive capabilities are limited, because the only types are null, bo
olean, numeric, string, array, and object.
MongoDB adds support for a number of additional data types while keeping JS
ON s essential key/value pair nature.
* null
{"x" : null}
* boolean
{"x" : true}
* 32-bit integer: This cannot be represented on the shell. JavaScript support
s only 64-bit floating point numbers, so 32-bit integers
will be converted into those.
* 64-bit integer: the shell cannot represent these. The shell will display th
em using a special embedded document
* 64-bit floating point number: All numbers in the shell will be of this type
. Thus, this will be a floating-point number:

{"x" : 3.14}
As will this:
{"x" : 3}
* string: UTF8 characters
{"x" : "foobar"}
* symbol: This type is not supported by the shell. If the shell gets a symbol
from the database, it will convert it into a string.
* object id: An object id is a unique 12-byte ID for documents
{"x" : ObjectId()}
* date: Dates are stored as milliseconds since the epoch. The time zone is no
t stored.
{"x" : new Date()}
* Regular Expresssion: Documents can contain regular expressions, using JavaS
cript s regular expression syntax:
{"x" : /foobar/i}
* code: Documents can also contain JavaScript code:
{"x" : function() { /* ... */ }}
* binary data: Binary data is a string of arbitrary bytes. It cannot be manip
ulated from the shell.
* maximum value: BSON contains a special type representing the largest possib
le value. The shell does not have a type for this.
* minimum value: BSON contains a special type representing the smallest possi
ble value. The shell does not have a type for this.
* undefined:
{"x" : undefined}
* array:
{"x" : ["a", "b", "c"]}
* embedded document:
{"x" : {"foo" : "bar"}}
*** Numbers: MongoDB has three number types (4-byte integer, 8-byte integer,
and 8-byte float).
*** _id and objectIds: In a single collection, every document must have a uni
que value for "_id".
The "_id" key s value can be any type, but it defaults to
an ObjectId.
ObjectIds use 12 bytes of storage,
-====0 1 2 3 | 4 5 6 | 7 8 | 9 10 11
Timestamp | Machine | PID | Increment
-====The first four bytes of an ObjectId are a timestamp in
seconds since the epoch.
This allows for up to 2563 (16,777,216) unique ObjectI
ds to be generated per process in a single second.
*** Autogeneration of _id: if _id not presented when document inserted, one w
ill be generated automatically.
This can be handled by the MongoDB server but will
generally be done by the driver on the client side.
================================================================================
===================================================================
3. Creating, Updating, and Deleting Documents
Insert/Batch Insert: When you perform an insert, the driver you are using converts the data struct
ure into BSON, which it then sends to the database
Documents larger than 4MB (when converted to BSON) cannot be saved to the dat
abase.

To see a size of doc execute Object.bsonsize(doc).

A batch insert is a single TCP request, meaning that you do not incur the ove
rhead of doing hundreds of individual requests.
e.g.:
objStudent = {"RollNo": 1, "Name": "Jon", "Date": new Date()}
db.Student.insert(objStudent);
db.Student.insert({"RollNo": 2, "Name": "Don", "Date": new Date()});
Delete: db.Student.remove({RollNo: 1}); (Based on criteria)
db.Student.remove({}); (all)
-- If there are million documents and all needs to be removed then drop of
collection is much faster.
Update: Update takes two parameters: a query document, which locates documents to upd
ate, and a modifier document
Note: Update actually takes four parameters, third parameter is true/false f
or upsert and 4th parameter is true/false for multi document update
but again that may be only possible with Modifiers (coming in next sec
tion).
Updates are atomic: if two updates happen at the same time, whichever one r
eaches the server first will be applied, and then the next one will be applied.
Thus, conflicting updates can safely be sent in rapid-fire succession without an
y documents being corrupted: the last update will win.
var st1 = db.Student.findOne();
st1.Name = "Son";
db.Student.update({RollNo: 1}, st1);
-----DB replacement on Shell: Collection users has one document.
Old: {"name": "joe", "friends": 32, "enemies": 2}
New: {"username": "joe",
"relationships":
{
"friends": 32,
"enemies": 2
}
}
var joe = db.users.findOne({"name": "joe"});
joe.relationships = {"friends": joe.friends, "enemies": joe.enemies}
joe.username = joe.name;
delete joe.enemies;
delete joe.friends;
delete joe.name;
db.users.update({"name" : "joe"}, joe);
Consider problem: ---Below three documents are having same name but
Collection people has following documents
{"_id" : ObjectId("4b2b9f67a1f631733d917a7b"),
{"_id" : ObjectId("4b2b9f67a1f631733d917a7c"),
{"_id" : ObjectId("4b2b9f67a1f631733d917a7d"),

different age and objectId.

"name" : "joe", "age" : 65}
"name" : "joe", "age" : 20}
"name" : "joe", "age" : 49}

joe = db.people.findOne({"name" : "joe", "age" : 20});

It will return below one.

{"_id" : ObjectId("4b2b9f67a1f631733d917a7c"), "name" : "joe", "age" : 20}
joe.age++;
db.people.update({"name" : "joe"}, joe);
Above will run into error due to three records with same name and it was exp
ecting to update only one record as 3rd and 4th
parameter not specified in update.
----Using Modifiers: Update modifiers are special keys that can be used to specify complex update
operations, such as altering, adding, or removing keys,
and even manipulating arrays and embedded documents. Always use $ operators
for modifying individual key/value pairs.
Note: When using modifiers, the value of "_id" cannot be changed.
"Inc" Increment Modifier ($inc): "$inc" increments the value of a key. If the key does not yet exist, it wi
ll be created with specified value.
Collection analytics has one document.
Document: {"url" : "www.example.com", "pageviews" : 52}
Update: db.analytics.update({"url" : "www.example.com"}, {"$inc" : {"pagev
iews" : 1}}) <It will increment pageviews (paveviews+1)>
Finding Doc: db.analytics.find()
O/P: {"url" : "www.example.com", "pageviews" : 53}
"Set" Modifier ($set) + ($unset): "$set" sets the value of a key. If the key does not yet exist, it will be
created.
Collection User has one document.
Document: {"_id" : ObjectId("4b253b067525f35f94b60a31"), "name" : "joe", "
age" : 30, "sex" : "male", "location" : "Wisconsin"}
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")}, {"
$set" : {"favorite book" : "war and peace"}})
<Above will add "key" favorite book with value.>
Update: db.users.update({"name" : "joe"}, {"$set" : {"favorite book" : "gr
een eggs and ham"}
<Above will update value for "key" favorite book>
Update: db.users.update({"name" : "joe"}, {"$set" : {"favorite book" : ["c
at's cradle", "foundation trilogy", "ender's game"]}})
<Above will update value for "key" favorite book (this time new values are
array)>
Update: db.users.update({"name" : "joe"}, {"$unset" : {"favorite book" : 1
}})
<Above will remove key from document>
To update embeded document refer to it's key as <main document key>.<embed
ed document key> : value.
Array Modifiers: An extensive class of modifiers exists for manipulating arrays.
"Push" Modifier ($push): "$push" adds an element to the end of an array if the specified key alr
eady exists and creates a new array if it does not.
It is to add single value. <Do not use it to add multiple array values

for key>
Collection User has one document.
Document: {"_id" : ObjectId("4b253b067525f35f94b60a31"), "name" : "joe",
"age" : 30, "sex" : "male", "location" : "Wisconsin"}
Insert: joe = {"_id" : ObjectId("4b253b067525f35f94b60a31"), "name" : "j
oe", "age" : 30, "sex" : "male", "location" : "Wisconsin"}
db.users.insert(joe);
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")},
{"$push": {"fav book": "power of now"}});
db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")},
{"$push": {"fav book": "new earth"}});
"Ne Modifier" in conjuction with push ($ne): Add a value to an array only if the value is not already present.
Update: db.users.update({"fav book": {"$ne" : "new earth"}},{"$push":
{"fav book": "new earth"}});
"AddToSet" Modifier ($addToSet): Use "$addToSet" to prevent duplicates in array. (Execute below statem
ent twice.)
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")
}, {"$addToSet": {"fav book": "new earth"}})
"Each" Modifier in conjuction with addToSet($each) (Add multiple values
at once in array): You can use "$addToSet" in conjunction with "$each" to add multiple u
nique values.
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")
}, {"$addToSet": {"fav book": {"$each" : ["earth1", "earth2"]}}})
"Pop" Modifier ($pop): Use "$pop", which can remove elements from either end.
{$pop : {key : 1}} : To remove elements from end of array.
{$pop : {key : -1}} : To remove elements from begining of array.
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")
}, {"$pop" : {"fav book": -1}})
"Pull" Modifier ($pull): Use "$pull" to remove elements of an array that match the given crite
ria.
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")
}, {"$pull" : {"fav book": "earth1"}})
Positional array modification: Array uses base index as 0 and put arrayIndex after key.
Update: db.users.update({"_id" : ObjectId("4b253b067525f35f94b60a31")
}, {"$set" : {"fav book.0": "abc"}})
Find position of array element using "$" and update particular positi
on (here avoding hardcoded array index).
db.users.update({"fav book" : "new earth1"}, { "$set" : {"fav book.$"
: "new earth"}});
Upsert: It follows syntax same as update, the third parameter to update specifies

that this should be an upsert.

Upsert: db.users.update({"url" : "/blog"}, {"$inc" : {"visits" : 1}}, true
)
For below scenario document will be inserted+updated using upsert and even
here collection is also created as it doesn't exists before.
Upsert: db.math.update({"count" : 25}, {"$inc" : {"count" : 3}}, true)
o/p: > db.math.find();
{ "_id" : ObjectId("552d65d59b09427c3c37bddc"), "count" : 28 }
The save Shell Helper: e.g.:
> var x = db.foo.findOne()
> x.num = 42
42
> db.foo.save(x)
> var x = db.foo.findOne()
Without save, the last line would have been a more cumbersome db.foo.update(
{"_id" : x._id}, x).
Updating Multiple Documents: 4th Parameter in update if true specifies that update all matched documents.
var student1 = {"RollNo" : 1, "Name" : "Amit", "Rank" : 2}
var student2 = {"RollNo" : 2, "Name" : "Amit", "Rank" : 3}
var student3 = {"RollNo" : 3, "Name" : "Amita", "Rank" : 3}
db.Student.insert(student1);
db.Student.insert(student2);
db.Student.insert(student3);
db.Student.update({"Name" : "Amit"},{ "$inc" : {"Rank" : 1}})
Above will update one document.
db.Student.update({"Name" : "Amit"},{ "$inc" : {"Rank" : 1}}, false, true)
Above will update two documents which matches criteria of "Name" = "Amit".
db.runCommand({getLastError : 1})
{"connectionId" : 9, "updatedExisting" : true, "n" : 2, "syncMillis" : 0, "w
rittenTo" : null, "err" : null, "ok" : 1}
[("n" : 2) + ("updatedExisting" : true)] means it has updated two documents.
Returning Updated Documents: (To be revisited)
================================================================================
===================================================================
4. Querying
Introduction to find: ================================================================================
===================================================================
5. Indexing
================================================================================
===================================================================
6. Aggregation
================================================================================
===================================================================
7. Advance Topics
================================================================================

===================================================================
8. Administrations
-> Starting and Stopping MongoDB: * Start: e.g.:
1) ./mongod
(It will
2) ./mongod
(It will
3) ./mongod
(It will

--port 5586 --fork --logpath mongodb.log

start mongodb service on port 5586)
start mongodb service by default on port 27017)
--rest
start http service by default on port 28017)

** Options: --dbpath: Default is /data/db (windows: c:\data\db).

If multiple mongod is running on one machine then each should
be running in different directory.
If attempt to start multiple mongod in one directory then belo
w error will come.
"Unable to acquire lock for lockfilepath: /data/db/mongod.lo
ck."
--port: Default port is 27017.
If plan to run multiple mongod on one machine then specify diffe
rent port for each.
--fork: It logs the info in path specified in logpath. <logpath is manda
tory with fork>
--logpath: Send all o/p to specified file rather than on command line.
If file specified not exists then it will create new one.
Check file permissions.
If old logs needs to be preserved then use logappend instead
of logpath.
--config: Use a config file for additional option not given on command l
ine.
FileBasedConfiguration:
e.g.: ./mongod --config /etc/.mongod.conf
./mongod -f /etc/.mongod.conf
Content of file (Text followed by # is comment)
-------------------------------------------------------# Start MongoDB as a daemon on port 5586
port = 5586
fork = true # daemonize it!
-------------------------------------------------------* Stop: Stop mongo db using CTRL+C, SIGINT (kill -2 <pid>) or SIGTERM(kill <
pid>)
When mongod receives SIGINT or SIGTERM it will do clean shutdown. It
will wait for currently running operation or
file preallocation to finish, close all open connections, flush all
data to disks, and halt.
** Another way to shutdown: > use admin
> db.shutdownServer();
-> Monitoring: ** Admin interface: http://localhost:28017/
Bydefault server listens port 1000 higher than the nat
ive driver port.

To turn on: ./mongod --rest

To turn off: ./mongod --nohttpinterface
This interface gives access to assertion, locking, ind
exing, replication, etc.
http://localhost:28017/l_status <raw server status inf
o>
** Server Status: db.runCommand({"serverStatus" : 1})
Server status provides detailed look on what is going in
side mongodb.
Info like uptime, server version, # of connections, glob
allock, etc.
All of the counters in the output are tracked from the t
ime server was started and will eventually
roll over if the counts get high enough. When a rollver
occurs for any counter, all counters will roll over,
and the value of "rollovers" in the "asserts" document w
ill increment.
** mongostat: mongostat prints some of the most important information avai
lable from serverStatus. A new line every second.
** Thirdparty plugins: Mongodb plug-ins exists for Nagios, Munin, Ganglia
and Cacti.
-> Security and Authentication: MongoDB supports per connection authentication,
albeit with a pretty coarse grained permission scheme.
** Authentication basics: When security is enabled, only authenticated use
r will be able to do read/write operation.
<Start the server with -auth command line option
to enable security
e.g.: ./mongod --auth>
MongoDB user belongs to database hence user crea
ted in one database may not workable for other.
MongoDB treats one database as special: admin.
A user in the admin database can be thought of a
s a superuser.
## Add admin user(user root has all rights)
> use admin
> db.addUser("root", "mongo123")
## Add user who can only read (here third parameter is "read_only=true
")
> use test
> db.addUser("support_reader", "opr123", true);
Note: AddUser method can be used to change user's password or readonly
state by simply calling it and supplying new password/state.
The 'addUser' shell helper is DEPRECATED. Please use 'createUser
' instead
## Connect using user created.
> use test
> db.auth("support_reader","opr123");
** How authentication works: Users of a given database are stored as docum
ent in its system.users collection.
## Retrieving info of stored users.
> use admin
> db.system.users.find()
## Removing existing users.
> db.system.users.remove({"user" : "reader"});
** Other security consideration
## BindIP: starts mongo with bindip and it will allow you to specify a
local IP address that mongod will be bound to.
<./mongod --bindip>
to disable http adming interfact starts mongod with nohttpin

terface
<./mongod --nohttpinterfact>
to disallow server side javascript execute starts mongod wit
h noscripting
<./mongod --noscripting>
================================================================================
===================================================================