Вы находитесь на странице: 1из 38

8/9/2017 High Scalability -

Hands down the best ever 25,000 year old selfie


from Pech Merle cave in southern France. (The Ice
Age)

If you like this sort of Stuff then please support


me on Patreon.

35%: US traffic is now IPV6; 10^161: decision


points in no-limit Texas holdem; 4.5 billion:
Facebook translations per day; 90%: savings by
moving to Lambda; 330TB: IBM's tiny tape
cartridge, enough to store 330 million books;
$108.9 billion: game revenues in 2017; 85%: of all
research papers are on Sci-Hub; 1270x: iPhone 5
vs Apollo guidance computer; 16 zettabytes: 2017
growth in digital universe;

Quotable Quotes:
Andrew Roberts: [On Napoleon] No
aspect of his command was too small to
escape notice.
Jason Calacanis: The world has
trillions of dollars sitting in bonds,
cash, stocks, and real estate, which is
all really dead money. It sits there
and grows slowly and safely, taking no
risk and not changing the world at all.
Wouldnt it be more interesting if we
put that money to work on crazy
experiments like the next Tesla,
Google, Uber, Cafe X, or SpaceX?

http://highscalability.com/ 4/41
8/9/2017 High Scalability -

@icecrime: The plural of its not a bug,


its a feature is its not a bug tracker, its
a backlog.
Jeff Darcy: When greater redundancy
drives greater dependency, its time to
take a good hard look at whether the net
result is still a good one.
uhnuhnuhn: "They ran their business into
the ground, but they did it with such great
tech!"
Angls-Alczar: Its very interesting to
think of our galaxy not as some isolated
entity, but to think of the galaxy as being
surrounded by gas which may come from
many different sources. We are
connected to other galaxies via these
galactic winds.
@ojiidotch: Main app now running Python
3.6 (was 2.7 until yesterday). CPU usage
40% down, avg latency 30% down, p95
60% down.
Nemanja Mijailovic: Its really difficult to
catch all bugs without fuzzing, no matter
how hard you try to test your software.
SandwichTeeth: a lot of companies have
security teams solely to meet audit
requirements. If you find yourself on a
team like that, you'll be spending a lot of
time just gathering evidence for audits,
remediating findings and writing policy. I
really loved security intellectually, but in
practice, the blue-team side of things
wasn't my cup of tea.
jph: security is needed to gradually
escalate a user's own identity verification
-- think of things like two-factor auth and
multi-factor auth, that can phase in (or
ramp up) when a user's actions enter a
gray area of risk. Some examples: when
a user signs in from a new location, or a
user does an especially large money
transfer, or a user resumes an account
that's been dormant for years, etc.
@hichaelmart: So while Google is
doubling down on gRPC it seems that
Amazon is going all in with CBOR. DDB
DAX uses some sort of CBOR-over-
sockets AFAICT

http://highscalability.com/ 5/41
8/9/2017 High Scalability -

Wysopal: Id like to see someone fixing


this broken market [insecure software and
hardware market]. Profiting off of that fix
seems like the best approach for a
capitalism-based economy.
Matthias Kppler: Microservices are often
intermediate nodes in a graph of services,
acting as faades where an incoming
request translates to N outgoing requests
upstream, the responses to which are
then combined into a single response
back downstream to the client.
Jack Fennimore: EA Play 2017 was
watchable the same way Olive Garden is
edible.
erikb: [On SoundCloud] TL;DR Top
Management started too late to think
about making actual money. They also
hired an asshole for their US offices.
When they got an opportunity to be
bought by Twitter they asked for way too
much money. And the CEO is basically
on a constant holidays trip since 2014,
while not failing to rub it in everybody's
face via Instagram photos.
Jennifer Mendez: If you dont have the
games people want to play, you can wave
goodbye to return on investment on a
powerful console. Does hardware matter?
Of course it does! But it doesnt matter if
you dont have anything to play on it.
Alex Miller: The utility of a blockchain
breaks down in a private or consortium
setting and should, in my opinion, be
replaced by a more performant engine
like Apache Kafka.
Krish: most of the multi-cloud usecases I
am seeing are about using different cloud
for different workloads. It could change
and I would expect them to embrace the
eventual consistency model initially
Ian Cutress: Then there is the Ryzen 3
1300X. Compared to the Core i3-
7300/7320 and the Core i5-7400, it
clearly wins on performance per dollar all
around. Compared to the Core i3-7100
though, it offers almost 5% more
performance for around $10-15 more,
which is just under 10% of the cost.
http://highscalability.com/ 6/41
8/9/2017 High Scalability -

throw2016: Just from an year ago the


cpu market has changed completely.
The sheer amount of choice at all
levels is staggering. For the mid level
user the 1600 especially is a
formidable offering, and the 1700 with
8 cores just ups the ante.
danmaz74: the main reason Rails is
declining in relevance isn't microservices
or the productivity (!) of Java, but the fact
that more and more development effort
for web applications is moving into JS
front-end coding.
Rohit Karlupia: we can deal with [S3]
eventual consistency in file listing
operations by repeating the listing
operation, detecting ghost and conceived
files and modifying our work queues to
take our new knowledge about the listing
status into account.
tboyd47: It's the end of an era. From 2005
to 2007, the "Web 2.0" craze, the release
of Ruby on Rails, and the rise of Agile
methods all happened at once. These
ideas all fed into and supported each
other, resulting in a cohesive movement
with a lot of momentum. The long-term
fact turned out to be that this movement
didn't benefit large corporations that have
always been and usually still are the main
source of employment for software
developers. So we have returned to our
pre-Rails, pre-agile world of high
specialization and high bureaucratic
control, even if Rails and "Agile" still exist
with some popularity.
@reneritchie: Only beginning to see the
advantages of Apple making everything
from atom to bit. Everything will be
computational.
Vasiliy Zukanov: switching to Kotlin will
NOT have any appreciable positive gains
on the cost, the effort or the schedule of
software projects
visarga: Over the years I have seen
astronomy become more like biology -
diverse both in the kinds of objects it
describes and their behavior.

http://highscalability.com/ 7/41
8/9/2017 High Scalability -

Jaana B. Dogan: I think the industry


needs a breakdown between product and
infra engineering and start talking how we
staff infra teams and support product
development teams with SRE. The
DevOps conversation is often not
complete without this breakdown and
assuming everyone is self serving their
infra and ops all the times.
David Rosenthal~ Does anybody believe
we'll be using Bitcoin or Ethereum 80
years from now?
Richard Jones: There is a physical lower
limit on how much energy it takes to carry
out a computation the Landauer limit.
The plot above shows that our current
technology for computing consumes
energy at a rate which is many orders of
magnitude greater than this theoretical
limit (and for that matter, it is much more
energy intensive than biological
computing). There is huge room for
improvement the only question is
whether we can deploy R&D resources to
pursue this goal on the scale thats gone
into computing as we know it today.

Don't miss all that the Internet has to say on


Scalability, click below and become eventually
consistent with all scalability knowledge (which
means this post has many more items to read so
please keep on reading)...

Click to read more ...

Todd Hoff | 4 Comments | Permalink | Share Article Print Article Email Article

in hot links

Like 2 people like this. Be the first of your friends.

Tweet

Wednesday
Aug022017

The Next Scalability Hurdle:


Massively Multiplayer
Mobile AR
WEDNESDAY, AUGUST 2, 2017 AT 9:30AM

http://highscalability.com/ 8/41
8/9/2017 High Scalability -

Many moons ago, in Building Super Scalable Systems:


Blade Runner Meets Autonomic Computing In The
Ambient Cloud, I said we still had scaling challenges
ahead, that we've not yet begun to scale, that we still
don't know how to scale at a planetary level.

That was 7 years ago. Now Facebook has 2 billion


monthly users. There's no reason to think they can't scale
an unimpressive 3.5x to handle the rest of the planet.
WhatsApp is at one billion daily users. YouTube is at 1.5
billion monthly users.

So it appears we do know how to service a whole planet


full of people (and bots). At least a select few companies
with vast resources know how. We are still no closer to
your average developer being able to field a planet scale
service. The winner take all nature of the Internet seems
to fend off decentralization like it's a plague. Maybe
efforts like Filecoin will change the tide.

There's another area we have scaling challenges:


Massively Multiplayer Mobile AR (Augmented Reality).
While AR has threatened to be the future for quite some
time, it now looks like the future may be just around the
virtual corner.

Apple Introducing ARKit, a hit with developers, means


that future will be sooner rather than later. One billion
iPhone users make it so. Remember when the iPhone
was introduced, how the increased data usage melted
AT&Ts' network? This will be worse.

Pokmon Go had a little event recently that shows what


incredible stress such systems will put on our
infrastructure. No need to repeat the story, iMore has it
all: Pokmon Go Fest: What happened and
why, Pokmon Go Fest's big flop shows Niantic needs to
think bigger, Pokmon Go Fest Chicago: The fun, the
failure, and the legendary, UPDATED: Are AT&T's iPhone
Problems Due to Network Configuration Errors?
http://highscalability.com/ 9/41
8/9/2017 High Scalability -

It's true, Pokmon Go has been well known for its scaling
problems, but this was a planned event, shouldn't it have
been handled better? No doubt. Still, a concentration of
20,000 players in a single shard, in such a small "kill
zone" like a park, is a challenge. Should they have
brought in Cell on Wheels, use high density WiFi, maybe
put up microwave links to increase the backhaul? Yep,
that seems reasonable. EM spectrum is a terrible thing to
waste.

But what happens when Pokmon Go Fest is just what


we call Tuesday? When everyone is using mobile AR?
Every product in every store, every building, every sign,
everything will have some sort of data driven overlay.
There will be no chance to build special infrastructure.
Infrastructure must be improved to handle the new loads.
Hopefully 5G will come to the rescue.

Spectrum isn't the only problem. Compute resources are


also a problem. Pokmon Go isn't a particularly data
intensive game. It doesn't require a lot interaction
between users or constant communication with backend
servers. What happens we we have multiple games like
that all operating at once?

Pokmon Go seems like a poster child for edge


computing. The entire shard could have been handled by
a portable onsite datacenter with its own local
communication infrastructure. An onsite datacenter
combines low latency compute with enough scale to
handle the load. My guess is the thundering herd problem
that blocked players connecting to the game would have
disappeared. Players would have connected quickly to
the local game servers and started playing the game with
little muss or fuss. Same with game state.

Perhaps in the future we'll have datacenter handoff


protocols just like we have cell network handoff protocols
today. And if we really do it right, the big scheduler in the
sky that will coordinate all these moving parts, might
consider distributed compute resources like smartphones
as part of the compute fabric.

We have not yet begun to scale Massively Multiplayer


Mobile AR.

Todd Hoff | 2 Comments | Permalink | Share Article Print Article Email Article

in AR, MMO, games

http://highscalability.com/ 10/41
8/9/2017 High Scalability -
Like 4 people like this. Be the first of your friends.

Tweet

Tuesday
Aug012017

Sponsored Post: Apple,


Domino Data Lab, Etleap,
Aerospike, Clubhouse,
Stream, Scalyr, VividCortex,
MemSQL,
InMemory.Net, Zohocorp
TUESDAY, AUGUST 1, 2017 AT 8:56AM

Who's Hiring?
Apple is looking for passionate VoIP
engineer with a strong technical
background to transform our Voice platform
to SIP. It will be an amazing journey with
highly skilled, fast paced, and exciting team
members. Lead and implement the
engineering of Voice technologies in Apples
Contact Center environment. The Contact Center
Voice team provides the real time communication
platform for customers interaction with Apples
support and retail organizations. You will lead the
global Voice, SIP, and network cross-functional
engineers to develop world class customer
experience. More details are available here.

Advertise your job here!

Fun And Informative Events


Advertise your event here!

Cool Products And Services


Enterprise-Grade Database Architecture. The
speed and enormous scale of todays real-time,
mission critical applications has exposed gaps in
legacy database technologies. Read Building
Enterprise-Grade Database Architecture for
Mission-Critical, Real-Time Applications to learn:
Challenges of supporting digital business
applications or Systems of Engagement;
Shortcomings of conventional databases; The
emergence of enterprise-grade NoSQL databases;
http://highscalability.com/ 11/41
8/9/2017 High Scalability -

Use cases in financial services, AdTech, e-


Commerce, online gaming & betting, payments &
fraud, and telco; How Aerospikes NoSQL database
solution provides predictable performance, high
availability and low total cost of ownership (TCO)

What engineering and IT leaders need to know


about data science. As data science becomes
more mature within an organization, you may be
pulled into leading, enabling, and collaborating with
data science teams. While there are similarities
between data science and software engineering,
well intentioned engineering leaders may make
assumptions about data science that lead to
avoidable conflict and unproductive workflows.
Read the full guide to data science for Engineering
and IT leaders.

Etleap provides a SaaS ETL tool that makes it easy


to create and operate a Redshift data warehouse at
a small fraction of the typical time and cost. It
combines the ability to do deep transformations on
large data sets with self-service usability, and no
coding is required. Sign up for a 30-day free trial.

InMemory.Net provides a Dot Net native in


memory database for analysing large amounts of
data. It runs natively on .Net, and provides a native
.Net, COM & ODBC apis for integration. It also has
an easy to use language for importing data, and
supports standard SQL for querying
data. http://InMemory.Net

www.site24x7.com : Monitor End User


Experience from a global monitoring network.

Working on a software product? Clubhouse is a


project management tool that helps software teams
plan, build, and deploy their products with ease. Try
it free today or learn why thousands of teams use
Clubhouse as a Trello alternative or JIRA
alternative.

Build, scale and personalize your news feeds and


activity streams with getstream.io. Try the API now
in this 5 minute interactive tutorial. Stream is free
up to 3 million feed updates so it's easy to get
started. Client libraries are available for Node,
Ruby, Python, PHP, Go, Java and .NET. Stream is
http://highscalability.com/ 12/41
8/9/2017 High Scalability -

currently also hiring Devops and Python/Go


developers in Amsterdam. More than 400
companies rely on Stream for their production feed
infrastructure, this includes apps with 30 million
users. With your help we'd like to ad a few zeros to
that number. Check out the job opening on
AngelList.

Scalyr is a lightning-fast log management and


operational data platform. It's a tool (actually,
multiple tools) that your entire team will love. Get
visibility into your production issues without juggling
multiple tabs and different services -- all of your
logs, server metrics and alerts are in your browser
and at your fingertips. . Loved and used by teams
at Codecademy, ReturnPath, Grab, and
InsideSales. Learn more today or see why Scalyr is
a great alternative to Splunk.

VividCortex is a SaaS database monitoring


product that provides the best way for
organizations to improve their database
performance, efficiency, and uptime. Currently
supporting MySQL, PostgreSQL, Redis,
MongoDB, and Amazon Aurora database
types, it's a secure, cloud-hosted platform that
eliminates businesses' most critical visibility
gap. VividCortex uses patented algorithms to
analyze and surface relevant insights, so users
can proactively fix future performance
problems before they impact customers.

MemSQL provides a distributed in-memory


database for high value data. It's designed to
handle extreme data ingest and store the data for
real-time, streaming and historical analysis using
SQL. MemSQL also cost effectively supports both
application and ad-hoc queries concurrently across
all data. Start a free 30 day trial here:
http://www.memsql.com/

Advertise your product or service here!

If you are interested in a sponsored post for an event,


job, or product, please contact us for more
information.

Click to read more ...

http://highscalability.com/ 13/41
8/9/2017 High Scalability -

Todd Hoff | Post a Comment | Permalink | Share Article Print Article Email

Article

in sponsored post

Like 2 people like this. Be the first of your friends.

Tweet

Friday
Jul282017

Stuff The Internet Says On


Scalability For July
28th, 2017s
FRIDAY, JULY 28, 2017 AT 8:56AM

Hey, it's HighScalability time:

Jackson Pollock painting? Cortical column? Nope,


it's a 2 trillion particle cosmological simulation using
4000+ GPUs. (paper, Joachim Stadel, UZH)

If you like this sort of Stuff then please support


me on Patreon.

1.8x: faster code on iPad MacBook Pro; 1 billion:


WhatsApp daily active users; 100 milliamps: heart
stopping current; $25m: surprisingly low take from
ransomware; 2,700x: improvement in throughput
with TCP BBR; 620: Uber locations; $35.5 billion:
Facebook's cash hoard; 2 billion: Facebook
monthly active users; #1: Apple is the world's most
profitable [legal] company; 500,000x: return on
destroying an arms depot with a drone;

http://highscalability.com/ 14/41
8/9/2017 High Scalability -

Quotable Quotes:
Alasdair Allan: Jeff Bezos statement that
theres not that much interesting about
CubeSats may well turn out to be the twenty
first centurys nobody needs more than
640kb.
@hardmaru: Decoding the Enigma with
RNNs. They trained a LSTM with 3000
hidden units to decode ciphertext with 96%+
accuracy.
@tj_waldorf: Morningstar achieved 97% cost
reduction by moving to AWS. #AWSSummit
Chicago
Ed Sperling: Moores Law is alive and well,
but it is no longer the only approach. And
depending on the market or slice of a market,
it may no longer be the best approach.
@asymco: With the end of Shuffle and Nano
iPods Apple now sells only Unix-enabled
products. Amazing how far that Bell Labs
invention has come.
@peteskomoroch: 2017: RAM is the new
Hadoop
Carlo Pescio: What if focusing on the
problem domain, while still understanding the
machine that will execute your code, could
improve maintainability and collaterally speed
up execution by a factor of over 100x
compared to popular hipster code?
@stevesi: Something ppl forget: moving
products to cloud, margins go down due to
costs to operate scale servicescosts move
from Customer to vendor.
@brianalvey: The most popular software
for writing fiction isn't Word. It's Excel.
@pczarkowski: How to make a
monolithic app cloud native: 1) run it in
a docker 2) change the url from .com
to .io
@tj_waldorf: Morningstar achieved
97% cost reduction by moving to
AWS. #AWSSummit Chicago
drinkzima: There is a huge general
misunderstanding in the profitability of
directing hotel bookings vs flight
bookings or other types of travel
consumables. Rate parity and high
commission rates mean that directing
http://highscalability.com/ 15/41
8/9/2017 High Scalability -

hotel rooms is hugely profitable and


Expedia (hotels.com, trivago, expedia)
and Priceline (booking.com) operate
as a duopoly in most markets. They
are both marketing machines that turn
brand + paid traffic into highly
profitable room nights.
Animats: This is a classic problem with AI
researchers. Somebody gets a good
result, and then they start thinking strong
human-level AI is right around the corner.
AI went through this with search,
planning, the General Problem Solver,
perceptrons, the first generation of neural
networks, and expert systems. Then
came the "AI winter", late 1980s to early
2000s, when almost all the AI startups
went bust. We're seeing some of it again
in the machine learning / deep neural net
era.
Charity Majors: So no, ops isn't going
anywhere. It just doesn't look like it used
to. Soon it might even look like a software
engineer.
@mthenw: As long as I need to pay for
idle its not serverless. Pricing is
different because in Lambda you pay for
invocation not for the runtime.
Kelly Shortridge: The goal is to make the
attacker uncertain of your defensive
environment and profile. So you really
want to mess with their ability to profile
where their target is
@CompSciFact: 'About 1,000 instructions
is a reasonable upper limit for the
complexity of problems now envisioned.' -
- John von Neumann, 1946
hn_throwaway_99: Few barriers to entry,
really?? Sorry, but this sounds a bit like
an inexperienced developer saying "Hey,
I could build most of Facebook's
functionality in 2 weeks." Booking.com is
THE largest spender of advertising on
Google. They have giant teams that A/B
test the living shite out of every pixel on
their screens, and huge teams of data
scientists squeezing out every last bit of
optimization on their site. It's a huge
barrier to entry.
http://highscalability.com/ 16/41
8/9/2017 High Scalability -

callahad: It's real [performance


improvements]. We've [Firefox] landed
enormous performance improvements
this year, including migrating most
Firefox users to a full multi-process
architecture, as well as integrating
parts of the Servo parallel browser
engine project into Firefox. There are
still many improvements yet-to-land,
but in most cases we're on track for
Firefox 57 in November.
Samer Buna: One important threat that
GraphQL makes easier is resource
exhaustion attacks (AKA Denial of
Service attacks). A GraphQL server can
be attacked with overly complex queries
that will consume all the resources of the
server.
wheaties: This is stupid. Really. Here
we are in a world where the
companies that own the assets (you
know, the things that cost a lot of
money) are worth less than the things
that don't own anything. This doesn't
seem "right" or "fair" in the sense that
Priceline should be a middleman,
unable to exercise any or all pricing
power because it does not control the
assets producing the revenue. I
wonder how long this can last?
platz: Apparently deep-learning and
algae are the same thing.
@CompSciFact: "If you don't run
experiments before you start designing a
new system, your entire system will be an
experiment." -- Mike Williams
Scott Aaronson: our laws of physics are
structured in such a way that even pure
information often has nowhere to hide: if
the bits are there at all in the abstract
machinery of the world, then theyre
forced to pipe up and have a measurable
effect.
The Internet said many more interesting
things this week. To read them all please
click through to the full article.

Cool interview with Margaret Hamilton--NASA's


First Software Engineer--on Makers. Programmers,
http://highscalability.com/ 17/41
8/9/2017 High Scalability -

you'll love this. One of the stories she tells is how


her daughter was playing around and selected the
prelaunch program during flight. That crashed the
simulator. So like a good programmer she wanted
to prevent this from happening. She tried to get a
protection put in because an astronaut could
actually do this during flight. Management would
certainly allow this, right? She was denied. They
said astronauts are trained never to make a
mistake so it could never happen. Eventually she
won the argument and was able to add code to
protect against human error. So little has changed
:-)

Don't miss all that the Internet has to say on


Scalability, click below and become eventually
consistent with all scalability knowledge (which
means this post has many more items to read so please
keep on reading)...

Click to read more ...

Todd Hoff | 2 Comments | Permalink | Share Article Print Article Email Article

in hot links

Like 2 people like this. Be the first of your friends.

Tweet

Tuesday
Jul252017

7 Interesting Parallels
Between The Invention Of
Tiny Satellites And Cloud
Computing
TUESDAY, JULY 25, 2017 AT 9:19AM

http://highscalability.com/ 18/41
8/9/2017 High Scalability -

CubeSats are revolutionizing space exploration because


they are small, modular, and inexpensive to build and
launch. On an episode of embedded.fm, Professor Jordi
Puig-Suari gives a fascinating interview on the invention
of the CubeSat. 195: A BUNCH OF SPUTNIKS.

What struck me in the interview is how the process of


how the CubeSat was invented parallels how the cloud
developed. They followed a very similar path driven by
many of the same forces and ideas.

Just what is a CubeSat? It's a "type of miniaturized


satellite for space research that is made up of multiples of
101010 cm cubic units. CubeSats have a mass of no
more than 1.33 kilograms per unit, and often use
commercial off-the-shelf (COTS) components for their
electronics and structure."

Parallel #1: University As Startup


Incubator
Click to read more ...

Todd Hoff | Post a Comment | Permalink | Share Article Print Article Email

Article

in cloud, satellite

Like 9 people like this. Be the first of your friends.

Tweet

Friday
Jul212017

Stuff The Internet Says On


http://highscalability.com/ 19/41
8/9/2017 High Scalability -

Stuff The Internet Says On


Scalability For July
21st, 2017
FRIDAY, JULY 21, 2017 AT 9:02AM

Hey, it's HighScalability time:

Ants Create Sinking, Eiffel Tower-Looking Structures

Afraid of AI? Fire ants have sticky pads so they can


form rafts, build towers, cross streams, & order
takeout. We can CRISPR these guys to fight Skynet.
(video, video, paper)

If you like this sort of Stuff then please support


me on Patreon.

222x: Bitcoin less efficient than a physical system


of metal coins and paper/fabric/plastic; #1: Python
use amongst Spectrum readers; 3x: time spent in
apps that don't make us happy; 1 million:
DigitalOcean users; 11.6 million: barrels of oil a day
saved via tech and BigData; 200,000: cores on
Cray super computer;$200B:
games software/hardware revenue by 2021; $3K:
for 50 Teraflops AMD Vega Deep Learning
Box; 24.4 Gigawatts: China New Solar In First Half
Of 2017;

Quotable Quotes:
sidlls: I think instead there is a category
error being made: that CS is an
appropriate degree (on its own) to
become a software engineer. It's like
suggesting a BS in Physics qualifies

http://highscalability.com/ 20/41
8/9/2017 High Scalability -

somebody to work as an engineer


building a satellite.
Elon Musk: AI is a fundamental existential
risk for human civilization, and I dont
think people fully appreciate that
Mike Elgan: Thanks to machine learning,
it's now possible to create a million
different sensors in software using only
one actual sensor -- the camera.
Amin Vahdat (Google): The Internet is no
longer about just finding a path, any path,
between a pair of servers, but actually
taking advantage of the rich connectivity
to deliver the highest levels of availability,
the best performance, the lowest latency.
Knowing this, how you would design
protocols is now qualitatively shifted away
from pairwise decisions to more global
views.
naasking: You overestimate AI.
Incompleteness is everywhere in CS.
Overcoming these limitations is not trivial
at all.
451: Research believes serverless is
poised to undergo a round of price cutting
this year.
Nicholas Bloom: We found massive,
massive improvement in performancea
13% improvement in performance from
people working at home
@CoolSWEng: "A Java new operation
almost guarantees a cache miss. Get rid
of them and you'll get C-like
performance." - @cliff_click #jcrete
DarkNetMarkets: We're literally funding
our own investigation.
Tristan Harris: By shaping the menus we
pick from, technology hijacks the way we
perceive our choices and replaces them
with new ones. But the closer we pay
attention to the options were given, the
more well notice when they dont actually
align with our true needs.
xvaier: If I have one thing to tell anyone
who is looking for business ideas to try
out their new programming skills on, I
strongly suggest taking the time to learn
as much as possible about the people to
whom you want to provide a solution,
http://highscalability.com/ 21/41
8/9/2017 High Scalability -

then recruiting one of them to help you


build it, lest you become another project
that solves a non-issue beautifully.
@sebgoa: Folks, there were
schedulers before kubernetes. Let's
get back down to earth quickly
Mark Shead: A finite state machine is a
mathematical abstraction used to design
algorithms. In simple terms, a state
machine will read a series of inputs.
When it reads an input it will switch to a
different state. Each state specifies which
state to switch for a given input. This
sounds complicated but it is really quite
simple.
xantrel: I started a small business that
started to grow, I thought I had to migrate
to AWS and increase my cost by 5xs
eventually, but so far Digital Ocean with
their hosted products and block storage
has handled the load amazingly well.
danluu: when Im asked to look at a
cache related performance bug, its
usually due to the kind of thing we just
talked about: conflict misses that prevent
us from using our full cache effectively6.
This isnt the only way for that to happen
bank conflicts and and false
dependencies are also common problems
Charles Hoskinson: People say ICOs
(Initial Coin Offering) are great for
Ethereum because, look at the price, but
its a ticking time-bomb. Theres an over-
tokenization of things as companies are
issuing tokens when the same tasks can
be achieved with existing blockchains.
People are blinded by fast and easy
money.
Charles Schwab: There don't seem to be
any classic bubbles near bursting at the
momentat least not among the ones
most commonly referenced as potential
candidates.
Sertac Karaman: We are finding that this
new approach to programming robots,
which involves thinking about hardware
and algorithms jointly, is key to scaling
them down.

http://highscalability.com/ 22/41
8/9/2017 High Scalability -

Michael Elling: When do people wake up


and say that weve moved full circle back
to something that looks like the hierarchy
of the old PSTN? Just like the circularity
of processing, no?
Benedict Evans: Content and access to
content was a strategic lever for
technology. Im not sure how much this is
true anymore. Music and books dont
matter much to tech anymore, and TV
probably wont matter much either.
SeaChangeViaExascaleOnDown:
Currently systems are still based around
mostly separately packaged processor
elements(CPUs, GPUs, and other)
processors but there will be an evolution
towards putting all these separate
processors on MCMs or Silicon
Interposers, with silicon interposers able
to have the maximum amount of parallel
traces(And added active circuitry) over
any other technology.
BoiledCabbage: Call me naive, but am I
the only one who looks at mining as one
of the worst inventions for consuming
energy possible?
Amin Vahdat (Google): Putting it
differently, a lot of software has been
written to assume slow networks. That
means if you make the network a lot
faster, in many cases the software cant
take advantage of it because the software
becomes the bottleneck.

Dropbox has 1.3 million lines of Go code, 500


million users, 500 petabytes of user data, 200,000
business customers, and a multi-exabyte Go
storage system. Go Reliability and Durability at
Dropbox. They use it for: RAT: rate limiting and
throttling; HAT: memcached replacement; AFS: file
system to replace global Zookeeper; Edgestore:
distributed database; Bolt: for messaging;
DBmanager: for automation and monitoring of
Dropboxs 6,000+ databases; Jetstream,
Telescope, block routing, and many more. The
good: Go is productive, easy to write and consume
services, good standard library, good debugging
tools. The less good: dealing with race conditions.

http://highscalability.com/ 23/41
8/9/2017 High Scalability -

Professor Jordi Puig-Suari talks about the invention


of CubeSat on embedded.fm. 195: A BUNCH OF
SPUTNIKS. Fascinating story of how thinking
different created a new satellite industry. The
project wasn't on anyone's technology roadmap,
nobody knew they needed it, it just happened. A
bunch of really bright students, in a highly
constrained environment, didn't have enough
resources to do anything interesting, so they
couldn't build spacecraft conventionally. Not
knowing what you're doing is an advantage in
highly innovative environments. The students took
more risk and eliminated redundancies. One
battery. One radio. Taking a risk that things can go
wrong. They looked for the highest performance
components they could find, these were
commercial off the shelf components that when
launched into space actually worked. The mainline
space industry couldn't take these sort of risks.
Industry started paying attention because the
higher performing, lower cost components, even
with the higher risk, changed the value proposition
completely. You can make it up with numbers. You
can launch 50 satellites for the cost of one
traditional satellite. Sound familiar? Cloud
computing is based on this same insight.
Modern datacenters have been created on
commodity parts and how low cost miniaturized
parts driven by smartphones have created whole
new industries. CubeSats' had a standard size, so
launch vehicles could standardize also, it didn't
matter where the satellites came from, they could
be launched. Sound familiar? This is the
modularization of the satellite launching, the same
force that drives all mass commercialization. Now
the same ideas are being applied to bigger and
bigger spacecraft. It's now a vibrant industry.
Learning happens more quickly because they get
to fly more. Sound familiar? Agile, iterative software
development is the dominant methodology today.

Don't miss all that the Internet has to say on


Scalability, click below and become eventually
consistent with all scalability knowledge (which
means this post has many more items to read so please
keep on reading)...

Click to read more ...

http://highscalability.com/ 24/41
8/9/2017 High Scalability -

Todd Hoff | Post a Comment | Permalink | Share Article Print Article Email

Article

in hot links

Like One person likes this. Be the first of your friends.

Tweet

Monday
Jul172017

You'll Never Believe The Big


Hairy Audacious Startup
John Jacob Astor Created
In 1808
MONDAY, JULY 17, 2017 AT 9:05AM

Think your startup has a Big Hairy Audacious Goal?


Along with President Thomas Jefferson, John Jacob
Astor conceived (in 1808), and implemented (in
1810) a plan to funnel the entire tradable wealth of the
westernmost sector of the North American continent
north of Mexico through his own hands. Early accounts
described it as the largest commercial enterprise the
world has ever known.

Think your startup raised a lot of money? Astor put up


$400,000 ($7,614,486 in today's dollars) of his own
money, with more committed after the first prototype
succeeded.

Think competition is new? John Jacob Astor dealt with


rivals in one of three ways: he tried to buy them out; if

http://highscalability.com/ 25/41
8/9/2017 High Scalability -

that didnt work, he tried to partner with them; if he failed


to join them, he tried to crush them.

Think your startup requires commitment? Joining Astor


required pledging five years of ones life to a start-up
venture bound for the unknownn.

Think your startup works hard? Voyageur's


paddled twelve to fifteen hours per day, with short breaks
while afloat for a pipe of tobacco. During that single day
each voyageur would make more than thirty thousand
paddle strokes. On the upper Great Lakes, the canoes
traversed hundreds of miles of empty, forested shorelines
and vast stretches of clear water without ports or
settlements or sails, except for the scattered Indian
encampment.

Think your product is complex? Astor planned, manned


and outfitted one overseas and two overland expeditions
to build the equivalent of a Jamestown settlement on the
Pacific Coast.

Think your startup parties hard? Every nook and corner in


the whole island swarmed, at all hours of the day and
night, with motley groups of uproarious tipplers and
whisky-hunters. It resembled a great bedlam, the frantic
inmates running to and fro in wild
forgetfulness. Many were eager for company and with a
yen to cut loosedrinking, dancing, singing, whoring,
fighting, buying knickknacks and finery from the beachs
shacks and stalls.

Think your startup was an adventure you can never


forget? I have been twenty-four years a canoe man, and
forty-one years in service; no portage was ever too long
for me. Fifty songs could I sing. I have saved the lives of
ten voyageurs. Have had twelve wives and six running
dogs. I spent all my money in pleasure. Were I young
again, I should spend my life the same way over. There is
no life so happy as a voyageurs life!

Think people at your startup dress weird? Above the


waist, the voyageurs wore a loose-fitting and colorful
plaid shirt, perhaps a blue or red, and over it, depending
on the weather, a long, hooded, capelike coat called a
capote. In cold winds they cinched this closed with a
waist sashthe gaudier the better, often red. From the
striking sash dangled a beaded pouch that contained
their fire-making materials and tobacco for their
inevitable pipe....The true Man of the North wore a

http://highscalability.com/ 26/41
8/9/2017 High Scalability -

brightly colored feather in his cap to distinguish himself


from the rabble.

Think your startup takes risks? Half of them died.

And like most startups, they accomplished a lot, but


ultimately failed to earn a payout.

Thomas Jefferson said to John Jacob Astor: Your name


will be handed down with that of Columbus & Raleigh, as
the father of the establishment and the founder of such
an empire. Unfortunately, not so much Tom. How many
have heard of Astor today? Not many, unless you've
traveled to Astoria, Oregon. Astoria in the right weather is
a gorgeous place with a hot beer scene.

It's trite to say the reward is in the journey, but in this


case the saying is true, the journey was larger than digital
life.

For the complete story read: Astoria: John Jacob Astor


and Thomas Jefferson's Lost Pacific Empire: A Story of
Wealth, Ambition, and Survival.

Todd Hoff | 1 Comment | Permalink | Share Article Print Article Email Article

Like 11 people like this. Be the first of your friends.

Tweet

Friday
Jul142017

Stuff The Internet Says On


Scalability For July
14th, 2017
FRIDAY, JULY 14, 2017 AT 9:04AM

Hey, it's HighScalability time:

http://highscalability.com/ 27/41
8/9/2017 High Scalability -

We've seen algorithms expressed in seeds. Here's


an algorithm for taking birth control pills expressed
as packaging. Awesome history on 99% Invisible.

If you like this sort of Stuff then please support


me on Patreon.

2 trillion: web requests served daily by Akamai; 9


billion: farthest star ever seen in light-years; 10^31:
bacteriophages on earth; 7: peers needed to repair
ransomware damage; $30,000: threshold of when
to leave AWS; $300K-$400K: beginning cost of
running Azure Stack on HPE ProLiant; 3.5M: files
in the Microsoft's git repository; 300M: Google's
internal image data training set size; 7.2
Mbps: global average connection speed; 85 million:
Amazon Prime members; 35%: Germany
generated its electricity from renewables;

Quotable Quotes:
Jessica Flack: I believe that science sits at
the intersection of these three things the
data, the discussions and the math. It is that
triangulation thats what science is. And
true understanding, if there is such a thing,
comes only when we can do the translation
between these three ways of representing
the world.

http://highscalability.com/ 28/41
8/9/2017 High Scalability -

gonchs: If your whole business relies on us


[Medium], you might want to pick a different
one
@AaronBBrown777: Hey @kelseyhightower,
if you're surfing GitHub today, you might find
it interesting that all your web bits come thru
Kubernetes as of today.
Psyblog: The researchers were surprised to
find that a more rebellious childhood nature
was associated with a higher adult income.
Antoine de Saint-Exupry: If you want to
build a ship, don't drum up people to collect
wood and don't assign them tasks and work,
but rather teach them to long for the endless
immensity of the sea.
Marek Kirejczyk: In general I would say: if
you need to debugyouve already lost your
way.
jasondc: To put it another way, RethinkDB did
extremely well on Hacker News. Twitter
didn't, if you remember all the negative posts
(and still went public). There is little relation
between success on Hacker News and
company success.
Rory Sutherland: What intrigues me about
human decision making is that there seems
to be a path-dependence involved - to which
we are completely blind.
joeblau: That experience taught me that you
really need to understand what you're trying
to solve before picking a database. Mongo is
great for some things and terrible for others.
Knowing what I know now, I would have
probably chosen Kafka.
0xbear: cloud "cores" are actually
hyperthreads. Cloud GPUs are single
dies on multi-die card. If you use GPUs
24x7, just buy a few 1080 Ti cards and
forego the cloud entirely. If you must use
TF in cloud with CPU, compile it yourself
with AVX2 and FMA support. Stock TF is
compiled for the lowest common
denominator
Dissolving the Fermi Paradox: Doing a
distribution model shows that even
existing literature allows for a
substantial probability of very little life,
and a more cautious prior gives a
significant probability for rare life
http://highscalability.com/ 29/41
8/9/2017 High Scalability -

Peter Stark: Crews with clique structures


report significantly more depression, anxiety,
anger, fatigue and confusion than crews with
core-periphery structures.
Patrick Marshall: Gu said that the team
expects to have a prototype [S2OSs
software-defined hypervisor is being
designed to centrally manage networking,
storage and computing resources] ready in
about three years that will be available as
open-source software.
cobookman: I've been amazed that more
people don't make use of googles
preemtibles. Not only are they great for
background batch compute. You can also use
them for cutting your stateless webserver
compute costs down. I've seen some people
use k8s with a cluster of preemtibles and non
preemtibles.
@jeffsussna: Complex systems cant be fully
modeled. Failure becomes the only way to
fully discover requirements. Thus the need to
embrace it.
Jennifer Doudna: a genomes size is not an
accurate predictor of an organisms
complexity; the human genome is roughly the
same length as a mouse or frog genome,
about ten times smaller than the salamander
genome, and more than one hundred times
smaller than some plant genomes.
Daniel C. Dennett: In Darwins Dangerous
Idea (1995), I argued that natural selection is
an algorithmic process, a collection of sorting
algorithms that are themselves composed of
generate-and-test algorithms that exploit
randomness (pseudo-randomness, chaos) in
the generation phase, and some sort of
mindless quality-control testing phase, with
the winners advancing in the tournament by
having more offspring.
Almir Mustafic: My team learned the
DynamoDB limitations before we went to
production and we spent time calculating
things to properly provision RCUs and
WCUs. We are running fine in production
now and I hear that there will be automatic
DynamoDB scaling soon. In the meantime,
we have a custom Python script that scales
our DynamoDB.
http://highscalability.com/ 30/41
8/9/2017 High Scalability -

I've written a novella: The Strange Trial of Ciri: The


First Sentient AI. It explores the idea of how a
sentient AI might arise as ripped from the headlines
deep learning techniques are applied to large social
networks. I try to be realistic with the technology.
There's some hand waving, but I stay true to the
programmers perspective on things. One of the big
philosophical questions is how do you even know
when an AI is sentient? What does sentience
mean? So there's a trial to settle the matter. Maybe.
The big question: would an AI accept the verdict of
a human trial? Or would it fight for its life? When an
AI becomes sentient what would it want to do with
its life? Those are the tensions in the story. I
consider it hard scifi, but if you like LitRPG there's a
dash of that thrown in as well. Anyway, I like the
story. If you do too please consider giving it a
review on Amazon. Thanks for your support!

Serving 39 Million Requests for $370/Month, or:


How We Reduced Our Hosting Costs by Two
Orders of Magnitude. Step 1: Just Go
Serverless: Simply moving to a serverless
environment had the single greatest impact on
reducing hosting costs. Our extremely expensive
operating costs immediately shrunk by two orders
of magnitude. Step 2: Lower Your Memory
Allocation: Remember, each time you halve your
functions memory allocation, youre roughly
halving your Lambda costs. Step 3: Cache Your
API Gateway Responses: We pay around $14 a
month for a 0.5GB API Gateway cache with a 1
hour TTL. In the last month, 52% (20.3MM out of
39MM) of our API requests were served from the
cache, meaning less than half (18.7MM requests)
required invoking our Lambda function. That $14
saves us around $240 a month in Lambda costs.

Don't miss all that the Internet has to say on


Scalability, click below and become eventually
consistent with all scalability knowledge (which
means this post has many more items to read so please
keep on reading)...

Click to read more ...

Todd Hoff | 3 Comments | Permalink | Share Article Print Article Email Article

in hot links

http://highscalability.com/ 31/41
8/9/2017 High Scalability -
Like 4 people like this. Be the first of your friends.

Tweet

Tuesday
Jul112017

Sponsored Post: Apple,


Domino Data Lab, Etleap,
Aerospike, Loupe,
Clubhouse, Stream, Scalyr,
VividCortex, MemSQL,
InMemory.Net, Zohocorp
TUESDAY, JULY 11, 2017 AT 8:57AM

Who's Hiring?
Apple is looking for passionate VoIP
engineer with a strong technical
background to transform our Voice platform
to SIP. It will be an amazing journey with
highly skilled, fast paced, and exciting team
members. Lead and implement the
engineering of Voice technologies in Apples
Contact Center environment. The Contact Center
Voice team provides the real time communication
platform for customers interaction with Apples
support and retail organizations. You will lead the
global Voice, SIP, and network cross-functional
engineers to develop world class customer
experience. More details are available here.

Advertise your job here!

Fun And Informative Events


Advertise your event here!

Cool Products And Services


Enterprise-Grade Database Architecture. The
speed and enormous scale of todays real-time,
mission critical applications has exposed gaps in
legacy database technologies. Read Building
Enterprise-Grade Database Architecture for
Mission-Critical, Real-Time Applications to learn:
Challenges of supporting digital business
applications or Systems of Engagement;
Shortcomings of conventional databases; The
emergence of enterprise-grade NoSQL databases;
http://highscalability.com/ 32/41
8/9/2017 High Scalability -

Use cases in financial services, AdTech, e-


Commerce, online gaming & betting, payments &
fraud, and telco; How Aerospikes NoSQL database
solution provides predictable performance, high
availability and low total cost of ownership (TCO)

What engineering and IT leaders need to know


about data science. As data science becomes
more mature within an organization, you may be
pulled into leading, enabling, and collaborating with
data science teams. While there are similarities
between data science and software engineering,
well intentioned engineering leaders may make
assumptions about data science that lead to
avoidable conflict and unproductive workflows.
Read the full guide to data science for Engineering
and IT leaders.

A note for .NET developers: You know the pain of


troubleshooting errors with limited time, limited
information, and limited tools. Log management,
exception tracking, and monitoring solutions can
help, but many of them treat the .NET platform as
an afterthought. You should learn
about Loupe...Loupe is a .NET logging and
monitoring solution made for the .NET platform
from day one. It helps you find and fix problems
fast by tracking performance metrics, capturing
errors in your .NET software, identifying which
errors are causing the greatest impact, and
pinpointing root causes. Learn more and try it free
today.

Etleap provides a SaaS ETL tool that makes it easy


to create and operate a Redshift data warehouse at
a small fraction of the typical time and cost. It
combines the ability to do deep transformations on
large data sets with self-service usability, and no
coding is required. Sign up for a 30-day free trial.

InMemory.Net provides a Dot Net native in


memory database for analysing large amounts of
data. It runs natively on .Net, and provides a native
.Net, COM & ODBC apis for integration. It also has
an easy to use language for importing data, and
supports standard SQL for querying
data. http://InMemory.Net

http://highscalability.com/ 33/41
8/9/2017 High Scalability -

www.site24x7.com : Monitor End User


Experience from a global monitoring network.

Working on a software product? Clubhouse is a


project management tool that helps software teams
plan, build, and deploy their products with ease. Try
it free today or learn why thousands of teams use
Clubhouse as a Trello alternative or JIRA
alternative.

Build, scale and personalize your news feeds and


activity streams with getstream.io. Try the API now
in this 5 minute interactive tutorial. Stream is free
up to 3 million feed updates so it's easy to get
started. Client libraries are available for Node,
Ruby, Python, PHP, Go, Java and .NET. Stream is
currently also hiring Devops and Python/Go
developers in Amsterdam. More than 400
companies rely on Stream for their production feed
infrastructure, this includes apps with 30 million
users. With your help we'd like to ad a few zeros to
that number. Check out the job opening on
AngelList.

Scalyr is a lightning-fast log management and


operational data platform. It's a tool (actually,
multiple tools) that your entire team will love. Get
visibility into your production issues without juggling
multiple tabs and different services -- all of your
logs, server metrics and alerts are in your browser
and at your fingertips. . Loved and used by teams
at Codecademy, ReturnPath, Grab, and
InsideSales. Learn more today or see why Scalyr is
a great alternative to Splunk.

VividCortex is a SaaS database monitoring


product that provides the best way for
organizations to improve their database
performance, efficiency, and uptime. Currently
supporting MySQL, PostgreSQL, Redis,
MongoDB, and Amazon Aurora database
types, it's a secure, cloud-hosted platform that
eliminates businesses' most critical visibility
gap. VividCortex uses patented algorithms to
analyze and surface relevant insights, so users
can proactively fix future performance
problems before they impact customers.

http://highscalability.com/ 34/41
8/9/2017 High Scalability -

MemSQL provides a distributed in-memory


database for high value data. It's designed to
handle extreme data ingest and store the data for
real-time, streaming and historical analysis using
SQL. MemSQL also cost effectively supports both
application and ad-hoc queries concurrently across
all data. Start a free 30 day trial here:
http://www.memsql.com/

Advertise your product or service here!

If you are interested in a sponsored post for an event,


job, or product, please contact us for more
information.

The Solution To Your


Operational Diagnostics Woes
Scalyr gives you instant visibility of your production
systems, helping you turn chaotic logs and system
metrics into actionable data at interactive speeds.
Don't be limited by the slow and narrow capabilities
of traditional log monitoring tools. View and analyze all
your logs and system metrics from multiple sources
in one place. Get enterprise-grade functionality with
sane pricing and insane performance. Learn more
today.

VividCortex Gives You Database


Superpowers
Database monitoring is hard, but VividCortex makes it
easy. Modern apps run complex queries at large
scales across distributed, diverse types of databases
(e.g. document, relational, key-value). The data tier
that these become is tremendously difficult to
observe and measure as a whole. The result?
Nobody knows exactly whats happening across all
those servers.

VividCortex lets you handle this complexity like a


superhero. With VividCortex, you're able to inspect all
databases and their workloads together from a bird's
eye view, spotting problems and zooming down to
individual queries and 1-second-resolution metrics in
just two mouse clicks. With VividCortex, you gain a
superset of system-monitoring tools that use only
global metrics (such as status counters), offering
deep, multi-dimensional slice-and-dice visibility into
queries, I/O, and CPU, plus other key measurements

http://highscalability.com/ 35/41
8/9/2017 High Scalability -

of your system's work. VividCortex is smart,


opinionated, and understands databases deeply, too:
it knows about queries, EXPLAIN plans, SQL bugs,
typical query performance, and more. It empowers
you to find and solve problems that you can't even
see with other types of monitoring systems, because
they dont measure what matters to the database.

Best of all, VividCortex isnt just a DBA tool. Unlike


traditional monitoring tools, VividCortex eliminates
the walls between development and production --
anybody can benefit from VividCortex and be a
superhero. With powerful tools like time-over-time
comparison, developers gain immediate production
visibility into databases with no need for access to
the production servers. Our customers vouch for it:
Problems are solved faster and they're solved before
they ever reach production. As a bonus, DBAs no
longer become the bottleneck for things like code
reviews and troubleshooting in the database.

Make your entire team into database superheros:


deploy VividCortex today. It only takes a moment,
and youre guaranteed to immediately learn
something you didnt already know about your
databases and their workload.

If you are interested in a sponsored post for an event,


job, or product, please contact us for more
information.

Click to read more ...

Todd Hoff | Post a Comment | Permalink | Share Article Print Article Email

Article

in sponsored post

Like One person likes this. Be the first of your friends.

Tweet

Friday
Jul072017

Stuff The Internet Says On


Scalability For July 7th, 2017
FRIDAY, JULY 7, 2017 AT 8:56AM

Hey, it's HighScalability time:

http://highscalability.com/ 36/41
8/9/2017 High Scalability -

What's real these days? I was at Lascaux II, an


exact replica of Lascaux. I was deeply, deeply
moved. Was this an authentic experience? A
question we'll ask often in VR I think.

If you like this sort of Stuff then please support


me on Patreon.

$400k: cost of yearly fake news


campaign; $50,000: cost to discredit a
journalist; 100 Gbps: SSDP DDoS amplification
attack; $5.97BN: wild guess on cost of running
Facebook on AWS; 2 billion: Facebook users; 80%:
Spotify backend services in production run as
containers; $60B: AR market by 2021; 10.4%: AMD
market share taken from Intel; 5 days: MIT drone
flight time; $1 trillion: Apple iOS
revenues; 35%-144%: reduction in image sizes; 10
petabytes: Ancestry.com data stored; 1 trillion:
photos taken on iPhone each year; $70B: Apple
App Store payout to developers; 355: pages in
Internet Trends 2017 report; 14: people needed to
make 500,000 tons of steel; 25%: reduced server-
rendering time with Node 8; 50-70%: of messages
Gmail receives are spam; 8,000: bugs found in
pacemaker code;

http://highscalability.com/ 37/41
8/9/2017 High Scalability -

Quotable Quotes:
Vladimir Putin: We must take into account the
plans and directions of development of the
armed forces of other countries Our
responses must be based on intellectual
superiority, they will be asymmetric, and less
expensive.
@swardley: What most fail to realise is that
the Chinese corporate corpus has devoured
western business thinking and gone beyond
it.
@discostu105: I am a 10X developer.
Everything I do takes ten times as long as I
thought.
DINKDINK: You grossly underestimate the
hashing capacity of the bitcoin network. The
hashing capacity, at time of posting, is
approximately 5,000,000,000
Gigahashes/second[1]. Spot measurement of
the hashing capacity of an EC2 instance is
0.4 Gigahashes/second[2]. You would need
12 BILLION EC2 instances to 51% attack the
bitcoin network.[3] Using EC2 to attack the
network is impractical and inefficient.
danielsamuels && 19eightyfour~
Machiavelli's Guide to PaaS: Keep your
friends close, and your competitors hosted.
Paul Buchheit: I wrote the the first version of
Gmail in a day!
@herminghaus: If you dont care about
latency, ship a 20ft intermodal container full
of 32GB micro-SD cards across the globe.
Its a terabyte per second.
@cstross: Okay, so now the Russian defense
industry is advertising war-in-a-can
(multimodal freight containerized missiles):
Dennett~ you don't need comprehension to
achieve competence.
@michellebrush~ Schema are
APIs. @gwenshap #qconnyc
Stacy Mitchell: Amazon sells more clothing,
electronics, toys, and books than any other
company. Last year, Amazon captured nearly
$1 of every $2 Americans spent online. As
recently as 2015, most people looking to buy
something online started at a search engine.
Today, a majority go straight to Amazon.
Xcelerate: I have noticed that Azure does
have a few powerful features that AWS and
http://highscalability.com/ 38/41
8/9/2017 High Scalability -

GCP lack, most notably InfiniBand (fast


interconnects), which I have needed on more
than one occasion for HPC tasks. In fact,
4x16 core instances on Azure are currently
faster at performing molecular dynamics
simulations than 1x"64 core" instance on
GCP. But the cost is extremely high, and I still
haven't found a good cloud platform for short,
high intensity HPC tasks.
jjeaff: I took about 5 sites from a $50 a month
shared cPanel plan that included a few
WordPress blogs and some custom sites and
put them on a $3 a month scaleway instance
and haven't had a bit of trouble.
@discordianfish: GCP's Pub/Sub is really
priced by GB? And 10GB/free/month? What's
the catch?
Amazon: This moves beyond the current
paradigm of typing search keywords in a box
and navigating a website. Instead, discovery
should be like talking with a friend who knows
you, knows what you like, works with you at
every step, and anticipates your needs. This
is a vision where intelligence is everywhere.
Every interaction should reflect who you are
and what you like, and help you find what
other people like you have already
discovered.
@CloudifySource: Lambda is always 100%
busy - @adrianco #awasummit #telaviv
#serverless
@codinghorror: Funny how Android sites
have internalized this "only multi core scores
now matter" narrative with 1/2 the CPU
speed of iOS hardware
@sheeshee: deleted all home directories
because no separation of "dev" &
"production". almost ran a billion euro site
into the ground with a bad loop.
We have quotes the likes of which even God
has never seen! Please click through to ride
all of them.

The Not Hotdog app on Silicon Valley may be a bit


silly, but the story of how they built the real app is
one of the best how-tos on building a machine
learning app you'll ever read. How HBOs Silicon
Valley built Not
Hotdog with mobile TensorFlow, Keras &
http://highscalability.com/ 39/41
8/9/2017 High Scalability -

React Native. The initial app was built in a weekend


using Google Cloud Platforms Vision API, and
React Native. The final version took months
of refinement. Google Clouds Vision API was
dropped because its accuracy in recognizing
hotdogs was only so-so; it was slow because of
the network hit; it cost too much. They ended
up using Keras, a deep learning library that
provides nicer, easier-to-use abstractions on top
of TensorFlow. They used on SqueezeNet due to its
explicit positioning as a solution for embedded
deep learning. SqueezeNet used only 1.25 million
parameters which made training much faster and
reduced resource usage on the device. What would
they change? timanglade: Honestly I think the
biggest gains would be to go back to a beefier, pre-
trained architecture like Inception, and see if I can
quantize it to a size thats manageable, especially if
paired with CoreML on device. Youd get the
accuracy that comes from big models, but in a
package that runs well on mobile. And this is
really cool: The last production trick we used was
to leverage CodePush and Apples relatively
permissive terms of service, to live-inject new
versions of our neural networks after submission to
the app store.

And the winner is: all of us. Serverless Hosting


Comparison: Lambda: Unicorn: $20,830.83. Heavy:
$120.16. Medium: $4.55. Light: $0.00; Azure
Functions: Unicorn: $19,993.60. Heavy: $115.40.
Moderate: $3.60. Light: $0.00; Cloud Functions:
Unicorn: $23,321.20. Heavy: $138.95. Moderate:
$9.76. Light: $0.00; OpenWhisk: Unicorn:
$21,243.20. Heavy: $120.70. Medium: $3.83. Light:
$0.00; Fission.io: depends on the cost of running
your managed Kubernetes cloud.

Minds are algorithms made physical. Seeds May


Use Tiny Brains to Decide When to Germinate:
The seed has two hormones: abscisic acid (ABA),
which sends the signal to stay dormant, and
gibberellin (GA), which initiates germination. The
push and pull between those two hormones helps
the seed determine just the right time to start
growing...According to Ghose, some 3,000 to 4,000
cells make up the Arabidopsis seeds...It turned out
that the hormones clustered in two sections of cells
near the tip of the seeda region the researchers
http://highscalability.com/ 40/41
8/9/2017 High Scalability -

propose make up the brain. The two clumps of


cells produce the hormones which they send as
signals between each other. When ABA, produced
by one clump, is the dominate hormone in this
decision center, the seed stays dormant. But as GA
increases, the brain begins telling the seed its
time to sprout...This splitting of the command
center helps the seed make more accurate
decisions.

Don't miss all that the Internet has to say on


Scalability, click below and become eventually
consistent with all scalability knowledge (which
means this post has many more items to read so please
keep on reading)...

Click to read more ...

Todd Hoff | 7 Comments | Permalink | Share Article Print Article Email Article

in hot links

Like 6 people like this. Be the first of your friends.

Tweet

http://highscalability.com/ 41/41