Desconstruindo a web

Willian Molinari (a.k.a PotHix)

Aviso

Willian Molinari
a.k.a PotHix

Plaev
.github.io

Porque dessa talk?

https://www.youtube.com/watch?v=nG1oGfrDiUc

https://www.youtube.com/watch?v=nG1oGfrDiUc

Essa talk se baseia em

Internet

Internet

The dawn of the net

Internet

http://pothix.com
http://pothix.com

http://pothix.com
http://pothix.com

http://pothix.com
http://pothix.com

uma URL?

O que significa PotHix

O que significa PotHix

http://pothix.com
http://pothix.com

uma URL!

http://pothix.com
http://pothix.com

Tem HSTS?

Strict-Transport-Security

http://pothix.com
http://pothix.com

Tem cache?

Expires

Cache-Control

Protocolo: http://
Domnio: pothix.com
Path: /
Tem cache de
DNS?

Chrome
Faz cache de DNS

http://aosabook.org/en/posa/high-performance-networking-in-chrome.html

getaddrinfo()

sistema
operacional

glibc
n
u

Im watching you!

glibc -> /etc/hosts

nss/nss_files/files-hosts.c

glibc -> getaddrinfo()

sysdeps/posix/getaddrinfo.c
2321 getaddrinfo (const char *name, const char *service,
2322
const struct addrinfo *hints, struct addrinfo **pai)
2323 {

getaddrinfo()
Tem nscd para
cache de DNS?

Internet

glibc
POSIX

Aplicao

Apresentao

Sesso

Transporte

Rede

Enlace

Fsico

Modelo
OSI

Aplicao

Apresentao

Sesso

Transporte

Rede

Enlace

Fsico

Modelo
OSI

S
R
L
Y
?

Aplicao

Apresentao

Sesso

Transporte

Rede

Enlace

Fsico

Aplicao

Transporte

Rede

Enlace

Fsico

Aplicao

Transporte

Rede

Enlace

Fsico

user
SO

Aplicao

DNS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

Aplicao

DNS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

cdigo disponvel em: chromium/src/net/dns

getaddrinfo()
__socket() & __connect()

sysdeps/posix/getaddrinfo.c
2515 fd = __socket (af, SOCK_DGRAM, IPPROTO_IP);
[...]
2526 [...] && __connect (fd, q->ai_addr, q->ai_addrlen) == 0

Syscalls in action! (*stripped)

terminal
$ sudo strace -f -e bind,socket,fork,connect -p $(ps aux | grep chromium | grep
-v nacl | awk '{ print $2}' | xargs echo | sed 's/ /,/g')
[...]
[pid 8739] socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 134
[pid 8739] connect(134, {sa_family=AF_INET6, sin6_port=htons(53), inet_pton
(AF_INET6, "2001:4860:4860::8888", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, 28) = -1 ENETUNREACH (Network is unreachable)
[pid 9010] socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 134
[pid 9010] connect(134, {sa_family=AF_INET, sin_port=htons(0),
sin_addr=inet_addr("192.30.252.154")}, 16) = 0
[pid 8739] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 134
[pid 8739] connect(134, {sa_family=AF_INET, sin_port=htons(80),
sin_addr=inet_addr("192.30.252.154")}, 16) = -1 EINPROGRESS (Operation now in
progress)
[...]

Syscalls in action! (*stripped)

terminal
$ sudo strace -f -e bind,socket,fork,connect -p $(ps aux | grep chromium | grep
-v nacl | awk '{ print $2}' | xargs echo | sed 's/ /,/g')
UDP para ipv6
[...]
nope :(
[pid 8739] socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 134
[pid 8739] connect(134, {sa_family=AF_INET6, sin6_port=htons(53), inet_pton
(AF_INET6, "2001:4860:4860::8888", &sin6_addr), sin6_flowinfo=0,
sin6_scope_id=0}, 28) = -1 ENETUNREACH (Network is unreachable)
[pid 9010] socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 134
UDP
[pid 9010] connect(134, {sa_family=AF_INET, sin_port=htons(0),
ipv4
sin_addr=inet_addr("192.30.252.154")}, 16) = 0
[pid 8739] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 134
[pid 8739] connect(134, {sa_family=AF_INET, sin_port=htons(80),
sin_addr=inet_addr("192.30.252.154")}, 16) = -1 EINPROGRESS (Operation now in
progress)
[...]
request

Happy eyebals (rfc6555): https://www.ietf.org/mailarchive/web/v6ops/current/msg22455.html

AAAA

sitev6
Browser

sitev4

de acordo com a rfc6555

AAAA

sitev6
Browser

sitev4

de acordo com a rfc6555

AAAA

sitev6
Browser

RST

de acordo com a rfc6555

sitev4

Aplicao

DNS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

UDP/IP
linux/blob/master/net/ipv4/udp.c

UDP

Contedo

Endereo (IP)

Aplicao

DNS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

Aplicao

DNS

Transporte

TCP / UDP

Rede

IP

Enlace

Mgica, bro!
Fsico

A visual explanation of how dns lookups work

ns1.dreamhost.com (66.33.206.206)
pothix.com -> 192.30.252.153
ns1.dreamhost.com (66.33.206.206)
pothix.com -> 192.30.252.154

A.ROOT-SERVERS.NET [.] (198.41.0.4)

a.gtld-servers.net [com] (192.5.6.30)

ns1.dreamhost.com [pothix.com]
(66.33.206.206) Got authoritative answer

Via dnstracer: `dnstracer -s . -4 -o pothix.com`

TCP/IP
UDP/IP

glibc
DNS

POSIX

Internet

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

Aplicao

HTTP/HTTPS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

Aplicao

HTTP/HTTPS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

TCP
linux/blob/master/net/ipv4/tcp.c

https://en.wikipedia.org/wiki/Transmission_Control_Protocol

ACK

CK

SYN + A
ACK

Three way handshake

SYN

SYNSENT

LISTENING

LISTEN

SYN+ACK

SYNSENT

SYN
LISTEN
RECEIVED

ACK

ESTABLISHED
SYNSENT

SYN
RECEIVED

 NIS!

ESTABLISHED
SYNSENT

NIS!

ESTABLISHED

Aplicao

HTTP/HTTPS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

HTTPS

TLS = fork(SSLv3)
SSLv3 is no longer supported in chrome

Aplicao

HTTP/HTTPS
TLS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

TLS

client hello
(CIPHERS, URL)

conexo estabelecida

Os primeiros milisegundos de uma conexo HTTPS

https://cc.dcsec.uni-hannover.de/

Server hello
(certificado, CIPHER)

conexo estabelecida

TLS_RSA_WITH_RC4_128_MD5

public
key

crypt

hash

Data vlida?
CA confivel?
Assinatura vlida?
URL esperada?

conexo estabelecida

conexo estabelecida

Gerei uma parada aqui

(pre_master secret)

conexo estabelecida
!^*@#$&!@(#$&!@#)$^$^
TLS_RSA_WITH_RC4_128_MD5

master secret

conexo estabelecida
!!@U#$N!Y*@#$*
TLS_RSA_WITH_RC4_128_MD5

 NIS!

NIS!

conexo estabelecida

TLS_RSA_WITH_RC4_128_MD5

criptografia
dos dados
verificao contra
contedo batizado

Internet
headers
url
cookies

HTTP

headers
url
cookies

TLS

HTTP

HTTP/HTTP2
chromium - net/http & net/spdy

Rafael Rinaldi
Dissecando o protocolo HTTP/2

HTTP 1x via telnet

terminal

$ telnet pothix.com 80
Trying 192.30.252.153...
Connected to pothix.com.
Escape character is '^]'.
GET /index.html HTTP/1.1
Host: pothix.com
HTTP/1.1 200 OK
Server: GitHub.com
...

Aplicao

HTTP/HTTPS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

Aplicao

Transporte

Rede

Enlace

Fsico

Aplicao

HTTP/HTTPS

Transporte

TCP / UDP

Rede

IP

Enlace

Ethernet / Wi-Fi

010101001011
Fsico

net/ipv4/tcp.c

include/net/cfg80211.h
net/wireless/nl80211.c
include/net/mac80211.h

drivers/net/wireless/iwlwifi/

papers/Vipin_Analysis_of_open_source_WLAN_driver_paper.pdf

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

a/b/g/n/ac

a/b/g/n/ac

C
S C
M A
A
arrier

ense

ultiple

ccess

ollision

voidance

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

TCP

meu router

# traceroute -T pothix.com
traceroute to pothix.com (192.30.252.153), 30 hops max, 60 byte packets
1 palantir (192.168.1.1) 8.718 ms 8.769 ms 8.868 ms
NET
2 10.18.128.1 (10.18.128.1) 25.058 ms 25.673 ms 27.189 ms
3 c8bd5001.virtua.com.br (200.189.80.1) 29.071 ms 29.073 ms 29.068 ms
4 embratel-T0-7-2-0-tacc01.spoph.embratel.net.br (200.178.127.57) 32.917 ms
embratel-T0-4-1-0-uacc03.spomb.embratel.net.br (189.42.182.37) 31.734 ms
embratel-T0-1-0-2-uacc04.spoph.embratel.net.br (201.56.189.9) 36.481 ms
5 ebt-H0-1-0-0-tcore01.spolp.embratel.net.br (200.230.1.242) 37.800 ms
200.244.212.73 (200.244.212.73) 37.803 ms
backbone
200.244.212.81 (200.244.212.81) 41.446 ms
embratel
6 ebt-BP1082-intl02.nyk.embratel.net.br (200.230.220.30) 204.822 ms
ebt-B11121-intl02.nyk.embratel.net.br (200.230.251.254) 192.381 ms
ebt-BP11521-intl02.nyk.embratel.net.br (200.230.220.174) 189.936 ms
7 fa-0-0-0.r23.nycmny01.us.bb.gin.ntt.net (129.250.202.185) 192.297 ms 172.074 ms 172.013 ms
8 ae-9.r22.asbnva02.us.bb.gin.ntt.net (129.250.2.149) 171.973 ms 167.012 ms 166.951 ms
9 ae-44.r06.asbnva02.us.bb.gin.ntt.net (129.250.6.113) 166.888 ms 166.873 ms
ae-45.r05.asbnva02.us.bb.gin.ntt.net (129.250.5.137) 212.669 ms
10 xe-0-3-0-18.r05.asbnva02.us.ce.gin.ntt.net (129.250.197.70) 202.448 ms
xe-0-9-0-16.r06.asbnva02.us.ce.gin.ntt.net (129.250.197.74) 201.593 ms 202.489 ms
11 pages.github.com (192.30.252.153) 171.337 ms * *

github pages

USA backbone

Internet

Nginx conf file

upstream backend {
server backend1.example.com
weight=5;
server backend2.example.com:8080;
server unix:/tmp/backend3;
server backup1.example.com:8080
server backup2.example.com:8080

TCP
Unix Socket

backup;
backup;

}
...

http://nginx.org/en/docs/http/ngx_http_upstream_module.html#server

Internet

http://12factor.net/port-binding

TCP
assets

HTTP parsing
ragel vindo l do velho mongrel*

*de acordo com o documento de design do unicorn

Internet

TCP
assets

Webserver -> Framework

baseado no PEP3333

Internet

TCP
assets

lib/rails/application.rb
Rack -> Rails

mais informaes no post do timaro no omniref

 um Rack application
...
# Implements call according to the Rack API. It simply
# dispatches the request to the underlying middleware stack.
def call(env)
env["ORIGINAL_FULLPATH"] = build_original_fullpath(env)
env["ORIGINAL_SCRIPT_NAME"] = env["SCRIPT_NAME"]
super(env)
end
...

Informaes do Rack
{"GATEWAY_INTERFACE"=>"CGI/1.1",

"HTTP_HOST"=>"localhost:9292",

"PATH_INFO"=>"/favicon.ico",

"HTTP_CONNECTION"=>"keep-alive",

"QUERY_STRING"=>"",

"HTTP_USER_AGENT"=>"Mozilla/5.0 (X11; Linux

x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/46.0.2490.71 Safari/537.36",

"REMOTE_ADDR"=>"127.0.0.1",
"REMOTE_HOST"=>"localhost.localdomain",
"REQUEST_METHOD"=>"GET",
"REQUEST_URI"=>"http://localhost:9292/favicon.
ico",
"SCRIPT_NAME"=>"",
"SERVER_NAME"=>"localhost",
"SERVER_PORT"=>"9292",
"SERVER_PROTOCOL"=>"HTTP/1.1",
"SERVER_SOFTWARE"=>"WEBrick/1.3.1 (Ruby/2.2.3
/2015-08-18)",

"HTTP_ACCEPT"=>"*/*",
"HTTP_REFERER"=>"http://localhost:9292/",
"HTTP_ACCEPT_ENCODING"=>"gzip, deflate, sdch",
"HTTP_ACCEPT_LANGUAGE"=>"en-US,en;q=0.8,pt-BR;q=0.
6,pt;q=0.4,es;q=0.2,eo;q=0.2", "rack.version"=>[1,
2],
"rack.multithread"=>true, "rack.multiprocess"
=>false, "rack.run_once"=>false, "rack.url_scheme"
=>"http", "HTTP_VERSION"=>"HTTP/1.1",
"REQUEST_PATH"=>"/favicon.ico"}

mais informaes no post do timaro no omniref

Rails::Engine
Todas as apps so Engines

mais informaes no post do timaro no omniref

Rack middlewares
[

ActionDispatch::Callbacks,

Rack::Sendfile,

ActiveRecord::Migration::CheckPending,

ActionDispatch::Static, Rack::Lock,

ActiveRecord::ConnectionAdapters::
ConnectionManagement,

#<ActiveSupport::Cache::Strategy::LocalCache::
Middleware:0x000000028fa168>,

ActiveRecord::QueryCache,

Rack::Runtime, Rack::MethodOverride,

ActionDispatch::Cookies,

ActionDispatch::RequestId,

ActionDispatch::Session::CookieStore,

Rails::Rack::Logger,

ActionDispatch::Flash,

ActionDispatch::ShowExceptions,

ActionDispatch::ParamsParser,

WebConsole::Middleware,

Rack::Head,

ActionDispatch::DebugExceptions,

Rack::ConditionalGet,

ActionDispatch::RemoteIp,

Rack::ETag

ActionDispatch::Reloader,

mais informaes no post do timaro no omniref

ActionDispatch
::Journey::Router
a primeira parte do seu cdigo a ser tocada
req.path_parameters
=> {:controller=>"pages", :action=>"home"}

mais informaes no post do timaro no omniref

ActionDispatch::Routing
::RouteSet::Dispatcher
achando o controller via rotas
ActiveSupport::Dependencies.constantize("#{params[:controller].camelize}Controller")
=> PagesController

mais informaes no post do timaro no omniref

ActionController::Metal
prepara o request e manda processar
def dispatch(name, request) #:nodoc:
[...]
process(name)

mais informaes no post do timaro no omniref

AbstractController::Base
encaminha para o seu controller
def process(action, *args)
[...]
process_action(action_name, *args) # que no fim s um `send`

mais informaes no post do timaro no omniref

V
M

Internet

M
V
C

TCP
assets

V
M

Internet

M
V
C

TCP
assets

V
M

Internet

M
V
C

TCP
assets

ActionDispatch
::Journey::Router
pega a Rack -> [status, header, body]

mais informaes no post do timaro no omniref

V
M

Internet

M
V
C

TCP
assets

V
M

Internet

M
V
C

TCP
assets

V
M

Internet

M
V
C

TCP
assets

V
M

Internet

M
V
C

TCP
assets

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

TCP/IP
UDP/IP

HTTP

glibc
DNS

POSIX

Internet

TCP/IP
UDP/IP

c-c-c-combo breaker

HTTP

glibc
DNS

POSIX

Internet

HTML

algoritmo de parsing
implementado no webkit do chrome

HEAD

exemplos retirados do post How browsers work

algoritmo de parsing
implementado no webkit do chrome

exemplos retirados do post How browsers work

HTML syntax error

<img src=http://pothix.com/avatar.gif>

thread

<img src=http://pothix.com/esperanto.png>

thread

src=http://pothix.com/fotodasferias.scr>
<img src=http://pothix.com/fotodasferias.jpg>

thread

<script src=http://pothix.com/marotagem.js>

sincrono

<script src=http://pothix.com/estiloso.css>

thread

<script src=http://pothix.com/javaescrito.js>

sincrono

<img src=http://pothix.com/avatar.gif>

thread

<img src=http://pothix.com/esperanto.png>

thread

src=http://pothix.com/fotodasferias.scr>
<img src=http://pothix.com/fotodasferias.jpg>

thread

<script src=http://pothix.com/marotagem.js>

sincrono

<script src=http://pothix.com/estiloso.css>

thread

<script src=http://pothix.com/javaescrito.js>

sincrono

exemplos retirados do post How browsers work

exemplos retirados do post How browsers work

Diego Eis: DOM, CSSOM e RenderThree - Introduo ao Browser Render Path

Resumindo
essa bagaa

glibc

Internet

V
M
M
V
C

Internet

No acredite em mgica
entenda como funciona de verdade
ceeeeerto.

Seja curioso
conhecimento nunca demais

Perguntas?
Willian Molinari
@PotHix
pothix@pothix.com

Slides: https://goo.gl/cC36RM

S
R
L
Y
?

The browser