` 120

A Comparative Study Getting Ready For ` 120

ISSN-2456-4885
On Various Open Source Remote Learning ISSN-2456-4885
Blockchain Platforms With FOSS

Volume: 08 | Issue: 09 | Pages: 108 | June 2020

What’s New In

DevOps
Building Gulp: A Understanding
The DevOps Based Continuous Integration
DevOps Tool For Web And Continuous
Pipeline Applications Delivery/Deployment
 India Technology Week @ Home

THANK YOU
EARLY INVESTORS
We thank the VISIONARY team members of these brands. They chose to believe
and invest in our concept of an Online Only Expo-cum-Conference.
They helped us create history. We'll henceforth call them Our Early Investors,
and will always remain grateful to them.

Visit ‘Online Only’ expo-cum-conference from the comfort of your HOME…

JUNE EDITION
17th – 19th JUNE, 2020

VISIT NOW: https://IndiaTechnologyWeek.com

For more information on sponsoring, exhibiting or attending, please call +91-966-774-3666 or team@indiatechnologyweek.com
CONTENTS JUNE 2020 ISSN-2456-4885

FOR U & ME
20 The Rise of AI and its Impact 15
34 How Technology is Helping
Fight Coronavirus
35 Role of Technology in
Maintaining Law and Order
FOCUS
40 The Five Best DevOps Tools
43 Understanding Continuous
Integration and Continuous
Delivery/Deployment
46 Building the DevOps Pipeline
with Jenkins
52 Understanding DevOps:
A Revolution in Software
Development
54 How DevOps Differs from
Traditional IT and Why
56 RCloud is DevOps for
Data Science
Getting Ready for Remote Learning with FOSS
58 How Prometheus Helps to
Monitor a Kubernetes Deployment
62 DevOps vs Agile: What You Should
26 31
Know About Both
65 Gulp: A DevOps Based Tool for
Web Applications

DEVELOPERS
89 Using spaCy for Natural Language
Processing and Visualisation
92 SPA JS: Building Cross-Platform
SPAs with Less Code

COLUMNS
70 CodeSport Breaking Down the Buzz Introduction to Green
ADMIN Around Quantum Computing Computing and its Importance
98 The Benefits of Using Terraform
as a Tool for Infrastructure-as-
Code (IaC)
OPEN GURU REGULAR FEATURES
103 Lighttpd: A Lightweight HTTP 07 FossBytes
Server for Embedded Systems

4  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

EDITOR
RAHUL CHOPRA

EDITORIAL, SUBSCRIPTIONS & ADVERTISING

DELHI (HQ)
D-87/1, Okhla Industrial Area, Phase I, New Delhi 110020

37 67
Ph: (011) 26810602, 26810603; Fax: 26817563
E-mail: info@efy.in

MISSING ISSUES
E-mail: support@efy.in

BACK ISSUES
Kits ‘n’ Spares
New Delhi 110020
Ph: (011) 26371661, 26371662
E-mail: info@kitsnspares.com

NEWSSTAND DISTRIBUTION
Ph: 011-40596600
E-mail: efycirc@efy.in

ADVERTISEMENTS
MUMBAI
Ph: (022) 24950047, 24928520
E-mail: efymum@efy.in

BENGALURU
Ph: (080) 25260394, 25260023 Six Things to Consider for a DevOps DevOps is the Future
E-mail: efyblr@efy.in
Transformationto the Cloud of Software Development
PUNE
Ph: 08800295610/ 09870682995
E-mail: efypune@efy.in

GUJARAT
Ph: (079) 61344948
E-mail: efyahd@efy.in
72 78
JAPAN
Tandem Inc., Ph: 81-3-3541-4166
E-mail: japan@efy.in

SINGAPORE
Publicitas Singapore Pte Ltd
Ph: +65-6836 2272
E-mail: singapore@efy.in

TAIWAN
J.K. Media, Ph: 886-2-87726780 ext. 10
E-mail: taiwan@efy.in

UNITED STATES
E & Tech Media
A Study of Various Open A Few Surprising Programming
Ph: +1 860 536 6677
E-mail: usa@efy.in
Source Blockchain Platforms Language Features

Printed, published and owned by Ramesh Chopra. Printed at Tara

Art Printers Pvt Ltd, A-46,47, Sec-5, Noida, on 28th of the previous
month, and published from D-87/1, Okhla Industrial Area, Phase I, New

83 94
Delhi 110020. Copyright © 2020. All articles in this issue, except for
interviews, verbatim quotes, or unless otherwise explicitly mentioned,
will be released under Creative Commons Attribution-NonCommercial
3.0 Unported License a month after the date of publication. Refer to
http://creativecommons.org/licenses/by-nc/3.0/ for a copy of the
licence. Although every effort is made to ensure accuracy, no responsi-
bility whatsoever is taken for any loss due to publishing errors. Articles
that cannot be used are returned to the authors if accompanied by a
self-addressed and sufficiently stamped envelope. But no responsibility
is taken for any loss or delay in returning the material. Disputes, if any,
will be settled in a New Delhi court only.

SUBSCRIPTION RATES
Year Newstand Price You Pay Overseas
(`) (`)
Five 7200 4320 —
Three 4320 3030 —
One 1440 1150 US$ 120 Image Feature Processing in Deep A Headless CMS: Delivering
Kindly add ` 50/- for outside Delhi cheques.
Please send payments only in favour of EFY Enterprises Pvt Ltd. Learning using Convolutional Pure Content in the Age of
Non-receipt of copies may be reported to support@efy.in—do mention
your subscription number. Neural Networks: An Overview Mobile-first Internet

6  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 FOSSBYTES
Microsoft offers up to GitHub and Major League
Hacking announce remote
US$ 100,000 to hack its open source internship
initiative
custom Linux OS GitHub and Major League Hacking
(MLH) have set up the MLH
Fellowship. This remote internship
programme will support the next
generation of developers with open
source projects. Facebook, Royal
Bank of Canada and DEV have
already signed on as supporters and
sponsors.
Mike Swift, CEO of MLH
said, “Enabling students to spend
their summer contributing to the
software that runs the world is
a unique opportunity for them.
They’ll work on meaningful
projects with their peers, under the
guidance of some of the world’s
most talented engineers. The
remote nature of this programme
Tech giant Microsoft is offering hackers up to US$ 100,000 if they can break the will democratise access to
security of the company’s custom Linux OS. The company had built a compact internships for countless students
and custom version of Linux for its Azure Sphere OS last year, which is designed worldwide.”
to run on specialised chips for its Internet of Things (IoT) platform. The OS has
been purpose-built for this platform to enable basic services and apps to run
isolated in a sandbox for security purposes.
Sylvie Liu, a security programme manager at Microsoft’s Security Response
Centre, said that the company will award a bounty of up to US$ 100,000 if the
Pluton security subsystem or the Secure World sandbox is breached in the Azure
Sphere OS. The bug bounty programme is part of a three-month research challenge
that runs from June 1 until August 31. 
Microsoft said that the challenge is focused on the Azure Sphere OS
itself, and not on the underlying cloud portion that’s already eligible for Azure
bounty programme awards. Azure Sphere was announced at last year’s Build
developer conference.
The programme will be for
Eclipse Foundation moves to Europe as part of its a 12-week period and will run
continued global expansion through the summer. It will train
The Eclipse Foundation has announced that in order to expand globally, it is up to 1,000 students full time on
establishing itself as a Europe based organisation. It said that through the creation of major open source projects under
Eclipse Foundation AISBL based in Brussels, it will be in a position to use its recent the mentorship of experienced
international growth and bring about global industry collaboration on open source engineers. The students will work
projects in strategic technologies. These include the cloud, edge computing, artificial in ‘pods’ of eight to ten students
intelligence, connected vehicles, telecommunications, and the Internet of Things. with open source projects and
The establishment of the new legal entity is expected to be finalised by July 2020. mentors. They will receive a
The Foundation said that it recognises the important role open source will play stipend for the programme through
in driving the digital and industrial transformations called for by the European GitHub sponsors.
Commission in its recent strategies. It added that contributions from a broad

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  |  JUNE 2020  |  7

FOSSBYTES
GitHub launches new
features and updates
GitHub has announced various
new features and updates in its
online Satellite 2020 event.  These
include GitHub Codespaces, GitHub
Discussions, code scanning, secret
scanning and GitHub Private Instances.
GitHub said that GitHub
Codespaces is a complete dev
environment within GitHub that lets
developers contribute immediately. It
said that every repository has its own
way of configuring a dev environment,
which often requires dozens of steps
before developers can write any code.
Sometimes, the environments of the
two projects they are working on,
conflict with one another.
GitHub Codespaces will give
developers a fully-featured cloud-
hosted dev environment that spins up
in seconds directly within GitHub,
company sources claim. Codespaces
can be configured by developers to
load their code and dependencies,
developer tools, extensions, and
dotfiles. The company said that
switching between the environments
has also been made simple and can
be navigated at any time. And when
switched back, the codespace is
automatically reopened.  enable them to buy parts for the VentMon to support more than 20 open source
Codespaces in GitHub includes
a browser based version of the full
VS code editor. It has support for
code completion and navigation,
extensions, terminal access, etc. If
developers prefer to use their desktop
IDE, they will be able to start a
codespace in GitHub and connect to
it from their desktop.
GitHub said that it has not
finalised the pricing for Codespaces
but also mentioned that the
code-editing functionality in the
codespaces IDE will always be free.
GitHub plans to offer simple pay-
as-you-go pricing similar to GitHub
Actions for computationally intensive
tasks such as builds. During the beta
phase, Codespaces is free.  Dor Laor, CEO and co-founder, ScyllaDB said, “Scylla 4.0 is our largest release
8  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
cross-section of European companies and governmental organisations to open
source projects will be crucial to ensure that these emerging technologies are fit for
Europe. These contributions will also need to see whether these technologies have
been designed with due consideration to the privacy and security of individuals and
organisations, as well as to the environment.
Juergen Mueller, chief technology officer and member of the executive board
of SAP SE said, “The Eclipse Foundation has a long track record of fostering
industry collaboration between global organisations and developers who share the
goal of creating scalable open source software. As a founding strategic member of
the Eclipse Foundation, SAP actively participates in several Eclipse projects and
working groups. With the legal move of the Eclipse Foundation to Brussels, we
expect more international collaboration across industries in an open environment.
We look forward to collaborating with members from around the world to create
and innovate together.”
Mozilla announces first three recipients of
COVID-19 Solutions Fund
Mozilla has announced the first three COVID-19 Solutions Fund recipients. A
month back, it had announced
the creation of a COVID-19
Solutions Fund as part of the
Mozilla Open Source Support
Program (MOSS). It aimed to
provide awards of up to US$
50,000 each to open source
technology projects working
towards reducing the spread of
the COVID-19 pandemic in some
way. The first three recipients
include VentMon, created by Public Invention in Austin, Texas; Recidiviz based in the
Bay area, and 3DBrooklyn for the COVID-19 Supplies NYC project.
VentMon can be used for testing of ventilator designs before deployment and
for ICU patient monitoring. The makers received a US$ 20,000 award, which will
engineering teams trying to build ventilators.
Recidiviz is a tech non-profit that has built a modelling tool which helps prison
administrators and government officials forecast the impact of COVID-19 on their
prisons and jails. The data equips them to assess changes they can make to slow the
spread, such as reducing the density in prison populations or granting early release
to people who are deemed to pose a low risk to public safety. The MOSS committee
approved a US$ 50,000 award for this recipient. Recidiviz’s tool was downloaded
by 47 states within 48 hours of launch.
ScyllaDB releases Scylla Open Source 4.0
ScyllaDB has announced the release of Scylla Open Source 4.0, a NoSQL database
for real-time Big Data workloads. The company said that Scylla has moved beyond
feature parity with Apache Cassandra. It is now serving as an open source drop-in,
no-lock-in alternative to Amazon DynamoDB.
Scylla Open Source 4.0 builds on Scylla’s close-to-the-hardware design.
The company said that it is written from the ground-up in C++ and delivers a
performance of millions of OPS on a single node. It scales out to hundreds of nodes,
and consistently achieves a 99 per cent tail latency of less than one millisecond.
ever, with improvements ranging from ease-of-use to new APIs, functionality and
 FOSSBYTES

performance. We welcome developers to discover the great performance Scylla

offers out of the box, at a reasonable price and without lock-in.” Nvidia plans to acquire
The company said that Scylla delivers better, more consistent performance Cumulus Networks to
along with a significant boost networking software
reduction in the total capabilities
cost of ownership (TCO) Nvidia has announced its plans to
and greater deployment acquire Cumulus Networks. The
flexibility. The production- company said that this will help boost
ready features in Scylla its networking software capabilities and
Open Source 4.0 offer launch a new era of the accelerated,
certain advantages over software-defined data centre. Recently,
Cassandra. As per the Nvidia had completed its acquisition of
company, these include Mellanox Technologies.
scaling up to any number of cores, streaming data to 60TB mega nodes, built-in In a blog post, the company
schedulers, unified cache, and self-tuning operations. stated, “With Cumulus, Nvidia can
innovate and optimise across the entire
Linux Foundation launches ToIP Foundation to boost data privacy networking stack, from chips and
Governments, non-profits, and the private sector, across the finance, health care systems to software including analytics
and enterprise software domains have joined hands with the Linux Foundation to like Cumulus NetQ, delivering great
improve universal security and privacy protocols for consumers and businesses in performance and value to customers.
the digital era. The ToIP Foundation is being developed with global, pan-industry This open networking platform is
support from organisations with sector-specific expertise. extensible, and allows enterprise and
Businesses today are struggling to protect and manage digital assets and data. cloud-scale data centres full control
This happens especially in an increasingly complex enterprise environment that over their operations.”
includes the Internet of Things (IoT), edge computing, artificial intelligence, etc. It
results in reducing the already low consumer confidence in how companies use or
profit from users’ personal
data, and is slowing
innovation on opportunities
like digital identity as well
as the adoption of other
new services.
The ToIP Foundation Cumulus is based in Mountain
will use digital identity View, California. It supports more
models that leverage than 100 hardware platforms with
interoperable digital wallets Cumulus Linux, its operating system
and credentials. It will also for network switches. Nvidia said
use the new W3C Verifiable Credentials standard to address these challenges and that its Nvidia Mellanox Spectrum
help consumers, businesses and governments to better manage risk, improve digital switches already ship with Cumulus
trust, and protect all forms of identity online. Linux and SONiC. This is the open
Jim Zemlin, executive director at the Linux Foundation said, “The ToIP source offering forged in Microsoft’s
Foundation promises to provide the digital trust layer that was missing in the Azure cloud and managed by the Open
original design of the Internet and to trigger a new era of human possibility. The Compute Project.
combination of open standards and protocols, pan-industry collaboration, and our The ONIE environment Cumulus
neutral governance structure will support this new category of digital identity and created is a software foundation for
verifiable data exchange.” Mellanox’s bare metal switches. Nvidia
added in the blog post, “Together,
Researchers unveil AI-powered open source platform for we built DENT, a distributed Linux
COVID-19 vaccine development software framework for retail and other
A team of machine learning, immunology, and bioinformatics researchers have enterprises at the edge of the network.
unveiled Epitopes.world. This is an AI-powered, open source and interactive Web And our Onyx operating system
platform to help accelerate vaccine development for COVID-19. continues to expand, especially in
The team is led by Tariq Daouda, PhD, who is currently a post- Ethernet Storage Fabrics (ESF).”
doctoral researcher at Harvard Medical School. It consists of volunteers

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  |  JUNE 2020  |  9

FOSSBYTES
UC Berkeley researchers
open source Reinforcement
Learning with Augmented
Data
As per a report by Venturebeat,
a group of researchers from the
University of California, Berkeley,
have open sourced Reinforcement
Learning with Augmented Data
(RAD). In an accompanying paper,
the authors said that this module can
improve any existing reinforcement
learning algorithm, and RAD can
achieve better compute and data
efficiency than Google AI’s PlaNet
and other cutting-edge algorithms
like DeepMind’s Dreamer and SLAC
from UC Berkeley.
The researchers said that
RAD achieved state-of-art results
on common benchmarks and
matched or beat every baseline
in terms of performance and data
efficiency across 15 DeepMind
control environments. It added
that it did this in part by applying
data augmentations for visual
observations. The co-authors of
the paper on RAD include Michael
‘Misha’ Laskin, Kimin Lee, and
Berkeley AI Research co-director
and Covariant founder Pieter Abbee.
As per the report, RAD was released
on preprint repository arXiv.
The paper said that, for the
first time, it can be shown that data
augmentations alone can significantly
improve the data-efficiency and
generalisation of RL methods
operating from pixels. This can
be done without any changes to
the underlying RL algorithm, on
the DeepMind Control Suite and
the OpenAI ProcGen benchmarks,
respectively. It added that by using
multiple augmented views of the
same data point as input, CNNs
are forced to learn consistencies in
their internal representations. This
results in a visual representation
that improves generalisation, data-
efficiency, and transfer learning.
10  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
who have doctoral degrees in machine learning and immunobiology,
bioinformaticians, and Web developers.
The process of developing a vaccine is typically a very lengthy and costly
one due to the number of virus-infected cells that need to be analysed. Dr Daouda
has developed an AI algorithm that can predict which parts of a virus are more
likely to be exposed at the surface of infected cells, which are called epitopes.
He did this while obtaining his doctorate degree at the Institute for Research in
Immunology and Cancer at the Université de Montréal.
These predictions can be used by researchers to generate a significantly shorter
list of potential targets to test in the creation of a vaccine. It reduces a process that
typically takes weeks or months, to just hours.
Authentic8 launches Open Source Intelligence Academy
Authentic8 has launched the Open Source Intelligence Academy (OSINT), which
will have an integrated suite of resources and tools available exclusively to its
customers. It aims to augment their training securely from home and other remote
work environments. The resources
will be delivered within Silo,
Authentic8’s Web isolation platform.
OSINT Academy will include the
OSINT Insider talk webcast series,
the OSINT Forum for collaboration
and peer-to-peer exchange, analysis
case studies, demonstration videos,
and OSINT Tradecraft Training. The
academy will provide access to a
range of resources designed to get
new analysts up to speed, streamline
their intelligence cycle, and equip
experienced researchers with new skills to stay current.
The curriculum was developed in partnership with training specialist OSINT
Combine to create more than 40 targeted training modules. It includes videos,
presentations, study guides and interactive quiz materials designed to build and
test analysts’ skillsets.
SUSE joins hands with iValue to offer open source
solutions for enterprises
Technology aggregator iValue InfoSolutions has announced that it is partnering
with SUSE to offer enterprise-grade, open source solutions for Linux, software-
 FOSSBYTES

defined infrastructure, and

application delivery. This will Cloud Foundry community
give enterprises greater control, and foundation to provide
flexibility, and cost efficiencies tutorial hub for new users
while undergoing digital The Cloud Foundry Foundation
transformation. has launched a hub for Cloud
Harsh Marwah, chief growth Foundry-related tutorials. This
officer at iValue InfoSolutions, will streamline the discovery and
said that the company is excited learning process for developers
to partner with SUSE. He added that iValue’s partnership with SUSE will help interested in learning more
partners and customers build sustainable, reliable, cost-effective, and secure about the family of open source
IT stacks based on open source technologies. iValue and SUSE aim to help projects. The tutorials have
customers transform their enterprises by tapping into open source innovations been created and curated by the
that offer no vendor lock-in, and are highly flexible, scalable and cost-effective. community, and will be provided
Rajarshi Bhattacharyya, country manager at SUSE India, said, “iValue’s in a free, simple way to learn
go-to-market strategy complements SUSE’s for its open source offerings. about Cloud Foundry.
SUSE is excited to partner with iValue to enable customers to realise their
digital transformation goals. This will be done by simplifying, modernising,
and accelerating their traditional and cloud-native applications across any IT
landscape in any environment.”

Facebook AI launches open source Chatbot named Blender

Facebook’s AI has built an open source open-domain chatbot called Blender as per
a blog post by the company. Engadget reported that Blender has been trained on 9.4
billion parameters, which is more than ten times as many as the previous largest OS
chatbots available on the internet.
Facebook said that Blender is the first chatbot to come with a diverse set As developers discover more
of conversational skills like empathy, knowledge and personality, all in one about the technology, they are
system. The blog stated that in terms of engagement, the bot feels ‘more human’ able to provide comments directly
according to human evaluators. It has been designed in such a manner that it can to the community -- about what
assume a persona, discuss nearly any topic, and show empathy in natural, 14-turn they have found valuable and
conversation flows. what is missing. This generates
a cycle of feedback that enables
the foundation to create new
materials to fill gaps in topics.
This feedback loop mimics the
technical contributions made to
open source projects. It builds
a collaborative ethos for Cloud
Foundry and other open source
communities.
Steve Greenberg, founder of
Resilient Scale said, “Resilient
Scale is happy to lend our services
by partnering with Cloud Foundry
and the community in order to
build this new resource. The
tutorial hub will make it easier
The tech giant also said the bot uses transformer neural networks on large for people unfamiliar with Cloud
amounts of conversational data. It uses previously available public domain Foundry to give it a try and reach
conversations that have 1.5 billion training examples of extracted conversations. out to enterprise developers
Facebook said that it has introduced Blended Skill Talk (BST) for training struggling with inefficient
and evaluating desirable skills linked to personality, knowledge, and the deployment environments.”
display of empathy.

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  |  JUNE 2020  |  11

FOSSBYTES
Venafi to acquire Jetstack
Venafi has announced a definitive
agreement to acquire open source
machine identity software provider,
Jetstack. The company has said that
this acquisition will transform the
way modern applications required by
digital transformation are secured.
Jeff Hudson, CEO of Venafi said,
“In the race to virtualise everything,
businesses need faster application
innovation and better security; both
are mandatory. Most people see these
requirements as opposing forces,
but we don’t. We see a massive
opportunity for innovation. This
acquisition brings together two
leaders who are already working
jointly to accelerate the development
process while simultaneously
securing applications against attack,
and there’s a lot more to do. Our
mutual customers are urgently asking
for more help to solve this problem
because they know that speed wins,
as long as you don’t crash.”
Venafi said that its innovative
solutions protect TLS, SSH, and
code signing machine identities for
the largest, most security-conscious
organisations and government
agencies in the world. Jetstack sources
have said that it supports and advises
enterprises using Kubernetes in
mission-critical infrastructure. The
company also fosters the cert-manager
open source community, which has
hundreds of code contributors and
millions of downloads.
The two companies have been
working together over the last two
years. They have been working to
accelerate the speed of innovation
for next-generation machine
identity protection in Kubernetes,
multi-cloud, service mesh and
microservices ecosystems.
12  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
mParticle launches open source developer toolset
Customer data platform mParticle has announced the beta release of a new
open source developer toolset. The company has said that this will give
engineering teams instant data quality protection and feedback in their integrated
development environments (IDE).
The toolset also comes with the Smartype feature. It translates any JavaScript
Object Notation (JSON) based data model into strongly-typed code. It also pairs
mParticle’s data planning application program interface (API), the command-line
interface (CLI), and static code analysis (linting).
In the earlier part of this year, mParticle released the next generation of its
Data Master product. This allows engineering, product and marketing teams to
collaboratively design and enforce a consistent data model. But once a development
team has created a data plan, developers still need an easy and fool-proof way to
implement the plan without having to leave their editor to cross-reference a Web UI.
According to company sources, this newly available data planning API will
allow engineering teams to use Data Master programmatically through an HTTP
API. Developers will be able to store data plans in their source code, and use their
own software development life cycle (SDLC) and approval processes to define the
data model to best suit their needs.
The company said that all constants called for in the data plan (event names,
attribute names and enum values) will be available as a machine-readable JSON
schema. Smartype will programmatically perform all CRUD operations on data
plans to decrease the time-to-data-quality and time to implement. Feedback
developers can use this to conform to a data plan.
The new CLI will offer an easy way to interface with the mParticle platform
through the command-line instead of the user interface, mParticle sources
claim. Developers can create, maintain, update, or delete data plans with version
control provided by a Git repository by using the CLI. The CLI also enables
developers to use linting tools to statically lint code against a data plan. This
helps to ensure adherence to their data plan, and that only high-quality data is
logged to their workspace.
AWS and Facebook team up for new open source
projects for PyTorch
Amazon Web Services and Facebook have jointly announced some new open
source projects for PyTorch. One is an open source machine learning framework
used to train artificial intelligence models.
FOSSBYTES

The new PyTorch projects announced include TorchServe and TorchElastic.

Altitude Angel launches The former is a model-serving framework for PyTorch that makes it easier for
open source hardware and developers to move new models into production. TorchElastic is a library that
software platform called developers can use to build fault-tolerant training jobs on Kubernetes clusters.
Scout PyTorch was created by Facebook’s AI research group to function as a
Altitude Angel, a provider of UTM machine learning library of functions for the Python programming language.
(unmanned traffic management) It has been designed for use with deep learning.
technology, has released an
open-sourced project named Dell launches set of open
Scout. It consists of hardware source networking solutions using SONiC
and firmware that will help drone Dell Technologies has announced a set of open source networking
manufacturers, software developers, solutions designed to simplify the management of data centres at scale.
and commercial drone pilots to The solutions, called Enterprise SONiC Distribution, by Dell Technologies
quickly connect to its global UTM, are built on Software for Open Networking in the Cloud (SONiC), an
the company reports. Altitude open source project headed by Microsoft.
Angel does not yet have plans to Tom Burns, senior VP and GM, Dell Technologies integrated products and
manufacture the device. It said that solutions said, “Our customers tell us that, while a hybrid cloud approach is
it is open to the possibility as part critical to their success, they struggle to maintain and scale their networks, and
of its work supporting the emerging manage them to effectively avoid multiple points of failure. By breaking switch
drone ecosystem. software into multiple, containerised components, we are providing enterprises
Scout is primarily intended for the means to drastically simplify the management of massive and complex
use in commercial and industrial networks and increase availability in a cloud model.”
drone applications. It comes with Dell has said that organisations are increasingly relying on modern hybrid
the capability to securely obtain and cloud models to do business. This leads to a monolithic and proprietary approach
broadcast a form of ‘network remote to networking, creating inefficiencies and unneeded complexity. According to
ID’. This is seen as a necessary Dell, its Enterprise SONiC Distribution removes complexity and creates a flexible
step for enabling routine drone use network through an approach built on open standards by integrating SONiC into
and ‘beyond visual line of sight’ the DNA of the Dell EMC PowerSwitch Open Networking hardware.
(BVLOS) flight. The hardware and
the firmware can be enhanced and
incorporated into a virtually limitless
set of scenarios due to its open
source nature.
Richard Parker, Altitude Angel,
CEO and founder said, “For
routine automated commercial and
industrial use of drones to take
flight, several challenges need
to be solved. Not least of these
challenges is, how will the drone be
connected to the digital air traffic
control systems of the future?
Some proprietary devices exist
which provide only a small part of
the solution -- getting the drone’s
location to a single UTM.”
He further said, “Scout represents Yousef Khalidi, corporate VP of Microsoft Azure Networking, Microsoft
one of the latest projects to emerge Corp., said, “SONiC is a leading open source network switch OS, empowering
from our R&D lab and we’re saying customers with modern and efficient cloud networking software. We’re pleased to
to users ‘take this device, experiment see Dell bringing enterprise support to its customers.”
with it, improve it and integrate it
with your solutions, and share your
findings with the community’.”
For more news, visit www.opensourceforu.com

14  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Insight For U & Me

Getting Ready
for Remote
Learning with

FOSS
The COVID-19 pandemic has made it mandatory to adapt to information and
communication technology tools to enable remote learning. This article explores various
free and open source tools that enable academia to cater to the needs of students. Tools
for various functions such as learning management systems (LMS), video conferencing,
building educational resources and evaluation are explored in this article.

T
he application of information
and communication
technology (ICT) has
been a topic of discussion
in academic circles for the past few
decades. However, with the sudden
disruption caused by the COVID-19
pandemic, ICT has emerged from
being a nice-to-have component into
becoming one that’s mandatory. Earlier,
the perception was that ICT could
enhance the teaching-learning process.
Now it is becoming a platform to enable
the teaching-learning process.
With the COVID-19-linked
lockdown, academia is now forced
to change its standard operating
procedures. It has become necessary for
teachers, students and parents to adapt
themselves to this sudden change with
the help of ICT tools, as these facilitate
uninterrupted continuation of the
teaching-learning process. This article
explores various free and open source
tools that enable academia to deliver
services in the best possible manner.
Why FOSS?
Though there are many proprietary
and paid software to enable the
teaching-learning process, they
involve recurrent licensing costs
that might not be affordable for all
in a diverse country like ours. One
of the most important factors in
selecting an ICT tool is inclusion.
We need to make sure every possible
learner is included. Free and open
source software (FOSS) is certainly
better in terms of inclusion because
it removes the cost of the software
from the scheme of things. Another
important advantage is that FOSS can
be customised to specific needs and
redistributed to the needy without the
need for any permissions.
Figure 1: Major tasks in remote learning
Is it only for video conferencing?
It’s no wonder that video conferencing
tools have suddenly become household
names with the onset of the COVID-19
pandemic. Teachers across the
globe have also started using video
conferencing tools to communicate
with their students. However, video
conferencing alone doesn’t constitute
the complete teaching-learning
process. It includes so many other
activities as well. Each of these
activities might require specific tools
and hence it is important to know the
right tools for each task. The major
tasks involved are listed below:
ƒƒ Building learning resources
ƒƒ Communicating with the learners
ƒƒ Conducting the evaluation process

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  15

For U & Me Insight
ƒƒ C
oordinating all the tasks to meet
learning objectives
Building learning resources
Learning resources form the core of any
teaching-learning process. Learning
content and how it is delivered will
determine the success or failure of the
teaching-learning process. Though there
are various types of learning resources,
for the sake of simplicity, let us classify
the resources that a teacher can build
for remote learning into the following
categories:
ƒƒ Presentations
ƒƒ Illustrations and mind maps
ƒƒ Video lectures
ƒƒ Podcasts
ƒƒ Interactive content
Let’s explore the tools belonging to
each of these categories.
Presentations: Presentations are the
most used method of delivering content.
The popular open source tools to build
presentations are listed below:
ƒƒ Impress (Libreoffice)
ƒƒ Beamer (LaTex)
ƒƒ Reveal.js
If you are already using some sort of
proprietary presentation software then
you might find shifting to LibreOffice
Impress very simple and effective.
Beamer is LaTex’s tool for
building presentations. If you are
teaching a subject that involves
Figure 2: Various categories of resources
Figure 3: Reveal.js browser based presentation
16  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
equations, mathematical notations or
algorithms, then Beamer will make
your presentations look elegant and
professional.
If you want to build browser
based presentations, then you should
give Reveal.js a try. If you have
introductory knowledge of HTML,
you can build presentations that run
inside the browser using Reveal.js. A
section of the code customised from
the official demo and its output (Figure
3) is shown below:
<div class=”slides”>
<section>
<h2>Open Source For You </h2>
<h3>Presentation on Remote Learning
Tools </h3>
<p>
<small>Created by <a href=”http://
kskuppusamy.in”>Dr. K.S. Kuppusamy </
a> </small>
<small>Created using <a href=””>
reveal.js </a> </small>
</p>
</section>
<section>
<h2>Reveal.js</h2>
<p>
Reveal.js makes the presentation
building so simple and effective.
</p>
</section>
There are powerful Web based
presentation frameworks such as
Impress.js. If you want to check out
Web based presentations, try exploring
https://github.com/impress/impress.js.
Mind maps: Mind maps make
explanations more effective. There are
many mind-mapping tools available
like FreeMind, FreePlane and Xmind
(Figure 4). Xmind provides many
features and makes the process simple.
Creating a video lecture: Video
lectures are an important component of
remote teaching. The sequence of steps
involved in building a screen-casting
based video lecture is as follows:
1. Prepare the presentation slides (using
Impress, Beamer or Reveal.js).
2. Use a screen-recording tool to
record the slides along with the
voice-over (Open Broadcaster
Software – OBS).
3. Use audio-editing tools to enhance
your audio recording (Audacity).
4. Use video-editing tools to edit
your video (OpenShot, Kdenlive,
Shotcut).
5. Upload the lecture to the Web or
LMS to share it with your students.
Step 1: This process is already
explained in the ‘Presentation’ section
of this article.
Step 2: This step involves using
screen recording software like Open
Broadcaster Software (OBS), which is
a powerful tool. Indeed, it’s not only for
screen recording but also has powerful
streaming capabilities that enable users
to set up various scenes, sources, etc.
You can include the inputs through a
webcam, screen contents, microphone,
etc. In my opinion, everyone who wants
to create a video lecture must spend
time on exploring OBS.
XMind
Mindmaps FreeMind
Freeplane
Figure 4: Mind mapping
 Insight For U & Me

#5.
Extract Audio From Video
Sha

reve

Imp
Bea .js

#1.
YouTube Link Sharing Do Noise Removal
re w

ress
als

Buil
Direct upload to LMS Equalize, Compress to Enhance

mer
Aud
Tools
ith t

dP
#3.
acit
Tools
he le

rese
Edit
y

ntat
arn

Aud
ers

ion
io
Building a Video Lecture

#2
#4.

. Re
Edit

co
Add Title

rd S
Vid
Out unncessary portions OBS

eo

cre
Add Captions

Rec
Ca d Scr
C

en
a
p
p

or
t
ture
ure een
Aud
Web
io
cam

Figure 5: The steps to building a video lecture

Crossplatform Kdenlive, which offer additional

Support for Formats
features.
3D animated titles
OpenShot Step 5: Once your video is
Curve based keyframe animation
Effects
ready, you can either upload it to the
Web (you can use a service such as
Configurable Interface
Titler
YouTube) and share the link with the
Many Effects and Transitions learners or if you don’t want to adopt
Open Source Video Editors Kdenline
Proxy Editing this method, you can directly upload it
Online Resources to your LMS itself.
Support for Various Formates Building interactive content:
Various audio effects
The lecture videos are unidirectional,
Video effects
ShotCut
i.e., they lack interactivity. Interactive
WebCam capture
Support for 4K
learning resources enable learners to
Network stream Playback interact with the content. Let’s explore
two services that enable interactivity:
Figure 6: Free and open source video editors ƒƒ TED-ED
ƒƒ H5P
Step 3: For a video lecture to track, which can be used for a podcast, TED-ED is a Web based service
be effective, the audio quality is a or an audio-only lecture. (https://ed.ted.com) to build interactive
key factor. As the teachers might be Step 4: This involves editing learning videos. To use TED-ED,
recording their lectures at their homes, the video, for which the FOSS tools just select an existing video, and add
there could be some sort of noise. The available are really impressive. The questions or prompts. This enables
audio quality can be greatly enhanced three important options are: reaction responses from the users. This
with the open source audio processing ƒƒ OpenShot (https://www.openshot. is a great tool that makes the content
software, Audacity. The audio from the org/) more interesting.
video file can be extracted by opening ƒƒ Kdenlive (https://kdenlive.org/en/) H5P (https://h5p.org/) stands
it in Audacity. Then the audio can be ƒƒ ShotCut (https://shotcut.org/) for HTML5 package. It is a free and
enhanced. The software has options to All these video editors are cross- open source content-collaboration
remove noise by identifying the noise platform. If you are a beginner, try framework. In other words, you can use
sample from the audio. Functions such exploring OpenShot, which is easy to it to build interactive videos, quizzes,
as Equalize and Compress can be used use. Even if you don’t have a higher find hotspots, etc.
to enhance the audio to sound like a configuration system, you should try
studio recording. Audacity can be used OpenShot. Once you are comfortable Evaluation tools
to record audio alone, or as a separate with it, try exploring ShotCut and Evaluation is an integral part of

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  17

For U & Me Insight
Figure 7: H5P illustrated
Figure 8: Evaluation tools
Figure 9: Jitsi – A free and open source video conferencing tool
the teaching-learning process. The
interactive content explained in the
earlier section can also be used as
evaluation tools at the micro level.
There are many open source evaluation
tools to conduct holistic evaluation.
Three of these are listed below:
ƒƒ Hot Potatoes (http://hotpot.uvic.ca/)
ƒƒ TCExam (https://tcexam.org/)
ƒƒ VirtualX (http://virtualx.
sourceforge.net/)
Hot Potatoes is available as
freeware. It can be used to build the
following types of tests: multiple-
choice, short-answer, jumbled-
sentences, crosswords, matching/
18  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
ordering and gap-filling.
The major features of TCExam
are that it is open source, platform-
independent, has community support,
accessibility (as per Web Content
Accessibility Guidelines that support
persons with disabilities), and the
capability to conduct paper testing
with OMR (optical mark recognition)
sheets, etc.
The various features of VirtualX
to conduct tests include the capability
to author and organise questions. It
has support for 12 different types of
questions, for formulas and equations,
as well as for graphical analysis.
Free and open source video
conferencing
Though there are many useful
proprietary tools available to conduct
video conferencing, open source
software always score better in terms of
privacy. One easy-to-use open source
tool for video conferencing is Jitsi. It
can be used in two different ways —
to make a custom installation or by
using the Jitsi Meet (https://meet.jit.
si/) service, which is a live version.
The latter is very simple and anyone
can start a meeting by simply entering
a meeting ID. There is no need for any
installation if you are using a desktop/
laptop. For smartphones, you have
to install the Jitsi app. Some of the
features of Jitsi are:
ƒƒ Browser based
ƒƒ No time limits
ƒƒ Capability to share screens
ƒƒ Option to stream in YouTube
ƒƒ Cloud based recording using
Dropbox
ƒƒ Integrated chat option
Jitsi is certainly worth exploring.
A video tutorial is available at https://
youtu.be/ymAtXVbotoU.
Open resources
It is not always a viable option to build
each learning resource from scratch.
There are many open educational
resources (OERs) available. Some of
the important OER repositories are
listed below.
ƒƒ OER Commons (https://www.
oercommons.org/): This is a public
digital library of open educational
resources. It has features to explore,
create and collaborate. The Open
Author tools make the creation of
resources simple by providing a
Web interface. If you have some
educational content to share, you
should explore Open Author.
ƒƒ MERLOT (https://www.merlot.
org/merlot/): This stands for
Multimedia Education Resource
for Learning and Online Teaching.
It provides access to a collection

of curated learning resources built
by an international community of
educators, researchers and learners.
ƒƒ NROER (https://nroer.gov.in/
welcome) is the National Repository
of Open Educational Resources. It
is an excellent initiative from the
government of India. It provides
open educational resources mapped
to school curricula, an e-library,
e-books and e-courses.
In addition to NROER, there are
other great initiatives from the Indian
government to promote technology
aided learning. Some of them are
shown in Table 1.
Learning management
systems (LMS)
As stated in the beginning of this
article, there is a need for coordinating
all the tasks associated with remote
learning. Learning management systems
(LMS) enable you to do this. There are
many choices in open source LMSs, as
listed below:
ƒƒ Moodle (https://moodle.org/)
ƒƒ ILIAS (https://www.ilias.de/en/)
ƒƒ SAKAI (https://www.sakailms.org/)
ƒƒ OpenOLAT (https://www.openolat.
com/?lang=en)
ƒƒ ATutor (https://atutor.github.io/)
In addition to these, there are certain
popular options available in freemium
mode such as Edmodo, Google
Classroom, etc. The choice of the
LMS depends on various factors such
as platform support, learner category,
types of features required, SCORM
compliance, etc.
Inclusion is the key
We need to do everything possible
to include every learner when we
are delivering learning content
using ICT tools.
ƒƒ There are a standard set of guidelines
for building resources, which can be
accessed by persons with disabilities.
Make sure to follow the Web
Content Accessibility Guidelines
(WCAG) to make your online
Insight For U & Me
Initiative URL
CEC (Consortium for Educational Communi- http://cec.nic.in/cec/
cation) (Swayam, Swayam Prabha and NME-
ICT are pioneering projects from CEC)
ePathshala http://epathshala.nic.in/
NPTEL https://nptel.ac.in/
Curricula for ICT in education https://ictcurriculum.gov.in/
Table 1
resources accessible to students an important element of classroom
with disabilities. management involves the use of visual
ƒƒ If some learners are facing and non-visual cues such as gauging
difficulties in accessing video the facial expressions of students,
resources due to the lack of stable assessing the mood of the class, etc.
network access, then we should In remote learning, when we present
provide resources in alternate content as recorded material, teachers
mediums such as in text form or cannot make use of these important
audio (which might be delivered feedback cues which makes teaching
using less bandwidth). Of course, more challenging. With the ongoing
there are still many other challenges COVID-19 pandemic, no one is sure
that would require detailed research how long social distancing will need
before providing solutions. to be enforced or whether it is going to
There are plenty of other tools that become the new normal.
have not been listed in this article. This is certainly an enormous
Enumerating all of them is neither challenge to academia. So it would be
possible in this article nor required. In wise to convert this challenge into an
my opinion, learning to use the tools opportunity to evolve a new ecosystem
is important but what is even more for teaching-learning. The most
crucial is the ability to adapt to the important thing is a change in mindset
current scenario. The skillsets required and the willingness to learn, unlearn
for remote learning are quite different and relearn among all the stakeholders,
from those needed for the live in- to deliver uninterrupted education to the
person teaching scenario. For example, next generation.
References
These links are in addition to those provided in the body of the article itself.
[1] Beamer, https://ctan.org/pkg/beamer?lang=en
[2] mpress, https://www.libreoffice.org/discover/impress/
[3] Reveal.js, https://revealjs.com/#/
[4] mpress.js, https://github.com/impress/impress.js
[5] FreeMind, http://freemind.sourceforge.net/wiki/index.php/Main_Page
[6] Freeplane, https://www.freeplane.org/wiki/index.php/Home
[7] XMind, https://www.xmind.net/
[8] OBS, https://obsproject.com/
[9] Audacity, https://www.audacityteam.org/
By: Dr K.S. Kuppusamy
The author is assistant professor of computer science, School of Engineering and
Technology, Pondicherry Central University. He has 14+ years of teaching and research
experience in academia and industry. His research interests include accessible
computing, Web information retrieval, mobile computing, etc.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  19
For U & Me Insight

The Rise of AI and

its Impact
There is no
accepted or
standard definition
of good artificial
intelligence (AI).
However, good
AI is one that
can help users
understand
various options,

(Credit: https://i0.wp.com/www.techregister.co.uk/wp-content)
explain tradeoffs
among multiple
possible choices
and then help
make those
decisions. Good AI
will always honour
the final decision
made by humans.

I
t is a common phenomenon that
if you repeat a word enough
number of times, it loses its
meaning. This is happening with
artificial intelligence (AI) already.
Although AI has finally made it to
the mainstream rather too quickly, its
journey is going to be rockier than it
was for other technologies in the past.
AI, as a concept, is not
anything new; it has been around
for centuries. However, it took
off significantly during the 1950s,
when Alan Turing explored it further.
Progress on it was, however, limited
due to the state of computer hardware
available at that time.
When computers became more
powerful in later years, these were
faster, affordable and had more power
in terms of storage as well as computing
speed. Since then, research in AI has
been growing steadily. There was a time
when we had merely 1MB memory
systems housed in a big box. Now,
we have 128GB memory systems in a
credit-card-sized device. Advancement
in hardware has significantly enabled
technological augmentation by
leaps and bounds.
In the past few years, there has
been sudden growth in all activities
related to AI, which was mainly
underpinned by the realisation of the
Internet of Things (IoT) and other
complementary technologies such as
Big Data, cloud computing, etc. And
since last year, we are seeing several
AI implementations.

20  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


There is no doubt that AI is still
in its childhood, but it has reached
a critical mass, where research as
well as application can happen
simultaneously. We can undoubtedly
say that we have changed gears.
AI is already making several
decisions that affect our life, whether
we like it or not, and has covered
significant ground in recent years.
AI is not everywhere yet
While it would be natural to think
that AI has penetrated almost every
single vertical or market, this is far
from the truth. At best, there are
only a few technology spot-fires in
a few select industries where AI is
making its mark.
Unfortunately, as always,
marketing gimmicks are at play
to make everyone feel that AI has
covered everything, while several
sections are still untouched.
Many image-recognition systems
are now better at detecting cancer
or micro-fractures from a patient’s
MRI or X-ray reports. Many pattern-
recognition systems can correlate
several pathological reports and make
AI-driven automated responders or chatbots
an almost precise prediction of the
health status of the patient. And yet,
medical recommendations without
a doctor’s explicit approval is not a
commonplace practice. And this is
good, because, when there is human
life at stake, systems should not make
a final call, ever. Therefore as far as
the medical field is concerned, AI
might only reach a status of assisted
intelligence and may not be permitted
(should not be allowed) to become a
mainstream phenomenon at all.
While companies are continually
taking humans out of the customer
service sector and replacing them with
chatbots or automated responders that
are AI-driven, human touch is becoming
expensive. At an event that saw startups
pitching their company/product, one
startup’s primary differentiation was
that it provided personal support for
all queries. Mostly, we are seeing an
exciting shift in terms of AI- and non-
AI-based offerings.
Self-learning applications is another
area where AI is making an entry.
Using customised learning, pace and
recommendations, it is becoming quite
popular. However, as that happens,
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  21
Insight For U & Me
teaching, coaching and mentoring
will soon become a high touch service
and still be in demand. Therefore it is
difficult to say whether AI has touched
this sector truly or just morphed it into
something else.
Another area where AI has not yet
touched and might not affect is live
entertainment and art. These are such
personalised and creative pursuits that
without having a human in it, they
would not have the same meaning.
There have been a few experiments
with AI creating art, but those art forms
have quite a different flavour to them.
AI systems can create art based on what
they have been trained for. Several
of those are mainly geometrical and
systematic shapes or pictures—nothing
that a human would necessarily draw
with a slightly acceptable and natural
imbalance in it. Real authorship of the
work of art cannot be yet bestowed to
an artificial system.
Creativity is some part process
and some part randomness, which is
the exact opposite of the rule-based
method. It is not likely for AI to be able
to contribute directly to the creative
industry any time soon.
 Your Favourite Website Has
IOT.electronicsforu.com
Secure | https://IOT.electronicsforu.com

Secure | https://electronicsforu.com

The Latest in IOT.

A platform for enablers, creators and providers of IOT solutions.

CAREERS.electronicsforu.com
Secure | https://CAREERS.electronicsforu.com

Jobs. Career. Technology.

Career advice and jobs related to electronics and IOT.
electronicsforu.com
TEST.electronicsforu.com
CHIPSNWAFERS.electronicsforu.com
Secure | https://TEST.electronicsforu.com

Secure | https://CHIPSNWAFERS.electronicsforu.com

It's All About Test & Measurement

Design. Electronics. Better.
Great resource for users, buyers & sellers of T&M solutions.
A resource for professional design engineers.

THANKS TO YOU—OUR ONLINE NETWORK IS

FACTS & FIGURES READERS EXPERTS

• Nine websites (two more coming soon) • You can access all content for FREE • Experts who want to share their
• Five major Facebook communities • You can subscribe to newsletters knowledge through articles,
• Seven major LinkedIN groups & pages for FREE --on most websites DIY Projects, etc are welcome
• Million-plus active users (monthly) • Register on our websites to get free invites • We also welcome experts who want to share
• Million-plus reach through Facebook to technical webinars and seminars their knowledge through webinars or seminars
• Fifty-thousand-plus industry connections through LinkedIN • You can contact us at editop@efy.in
Now Grown Into Many!
SMART.electronicsforu.com
Secure | https://SMART.electronicsforu.com

Profit from Technology

Enabling you to benefit from investments in technology.

BUSINESS.electronicsforu.com
Secure | https://BUSINESS.electronicsforu.com

Amazing DIY Projects. Latest Tech trends.

The hang-out for electronics enthusiasts. Business. Electronics. India.
Everything you want to know about India's electronics industry.

ACADEMIA.electronicsforu.com
DIRECTORY.electronicsforu.com
Secure | https://ACADEMIA.electronicsforu.com

Secure | https://DIRECTORY.electronicsforu.com

Everything related to Tech & Academia.

A platform for tech faculty and institutions. India. Electronics. Directory.
Enabling commerce between buyers & sellers of electronics in India.

AMONGST THE WORLD’S TOP 5 AND GROWING!

INDUSTRY
• You can advertise for as little as US$ 100 per month RESPONSE We now also act as marketing
partners for our clients and drive
• Special combo offers for advertisers in our print publications
• We’ve now enabled flexible CPM-based advertising
GUARANTEED entire marketing for them, where
we charge them on basis
• You can advertise on the platform of your choice (based on your target audience) SOLUTIONS of results and not efforts!
• We invite press releases at efy-edit-team@efy.in
• Press releases are published free of cost, subject to discretion of the editorial team
CONTACT US: Shrikant Rao • growmybiz@efy.in • +91-98111 55335
For U & Me Insight

Users and employees

have mixed reactions
As far as end-users of AI technology
are concerned, there is high-level fear,
uncertainty and doubt (FUD) amongst
the majority. The sheer duality of this
technology is a significant concern.
AI is a powerful tool, and just like any
other tool, humans can use it for good
or bad things. Moreover, since people
are not yet actively talking about how
to handle potential misuse of AI, this
has remained as a growing concern.
Another reason for having a
sceptical outlook towards AI is a
plausible fear of job losses. If there are
massive numbers of people losing jobs
without an alternative system in place,
it would be undoubtedly dangerous,
and this can create chaos.
But then again, if you think about
it deeply, you will realise that it is
not losing a job that concerns many.
What people usually worry about is
having nothing better to do when their
mainstream work is disrupted.
Unfortunately, majority of AI
implementation projects do not
address this issue upfront. Instead,
it is done as an afterthought. It is
perhaps the most substantial reason
for being sceptical of AI.
At a superficial level, many of us
appreciate the ease and convenience
these AI solutions are providing.
However, our comfort erodes as these
solutions start to increase their scope
and touch critical areas of our life, such
as banking, social benefits, security,
healthcare, jobs, driving and others.
Bias and racism have been front-
runners in the list of reasons for distrust
in AI. People also fear that AI may show
blatant disregard for human control.
This, however, does not have any
precedence, but it is practically possible
and, hence, is a legitimate concern.
Errors-at-scale is not a widely-
known issue, but those who have been
victims of this problem in the past see it
as one of the significant concerns when
using AI in daily life. Imagine when a
Use of AI in dermatology (Credit: https://www.airforcemedicine.af.mil)
public AI system cancels credit cards of
thousands of people because of some
error. The scale of chaos this may cause
is the main reason for this concern.
As a general observation, everyone
is comfortable as long as applications
are not touching or affecting core life
matters. They are comfortable in areas
of entertainment and luxury, but not so
much when critical aspects of life are
in the hands of an AI.
But enterprises have
different views
Despite mixed feelings and heightened
expectations from AI, the business
world still has some ability to see AI in
a relatively balanced manner. People
from a wide range of industries agree
that AI is tricky to deploy, and it could
suck a lot of money and time before
becoming useful. It can be costly, and
initial payout can be quite modest (and
sometimes lower than that). Overall
payback period for any AI project has
not been attractive, and in many cases, it
is hard to establish the same objectively.
Several experts find it unsettling
that some vendors are pushing AI
systems even before they figure
out the purpose and claim to know
what problems these will solve. Some
businesses discourage this approach, and
are taking a prudent view, but they are
in a minority. Most companies blindly
believe otherwise.
It is one thing to see breakthroughs
in gaming AI, such as in the game of
Go and Chess, or having devices that
turn on music at voice command. It
is another thing to use AI and make
changes in businesses, especially ones
that are not fundamentally digital.
When it comes to improving and
changing how businesses get done, AI
and other tools form only small cogs
of a giant wheel. Changes that bring
about company-wide repercussions are a
different ballgame.
Change management has not been
easy to handle in the past, and it is not
going to be any different in the future
either. Several experts from various
business domains need to be involved
for any significant change to occur,
and they have to be the best if we are
looking for effective outcomes. This
essentially means pulling the best people
from routine business work and letting
them focus on AI implementation.
This is a challenging proposition for
businesses of any size.

24  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


What the future holds
AI and other emerging technologies,
apart from bringing efficiencies, are
also bringing new possibilities. These
possibilities are creating new business
models and opportunities. This will
continue to happen in the future as
we progress.
Most daily tasks that depend on best
estimates or guesswork would also see a
significant shift due to abundance of data.
Due to access to more data, the need for
devices that can process this data on Edge
would increase and will be a key driver in
maintaining this progression.
One of the significant drivers
of these technological advances is
democratisation of resources. Whether
it is the Internet revolution, open source
hardware and software revolution,
or anything else, as AI technology
becomes a part of our daily lives, we
will see more of this democratisation
happening. This will be a crucial factor
and will keep boosting progress.
As of now, most AI applications
follow a supervised learning approach.
In years to come, we will start seeing
more and more of unsupervised
learning that will keep systems
updated continuously. However, this
will have one significant barrier to
cross, which is the trust factor. Unless
this trust factor improves, supervision
will remain a necessity.
There is no accepted or standard
definition of good AI. However,
good AI is one that can guide users
understand various options, explain
tradeoffs among multiple possible
choices and then help make those
decisions. Good AI will always honour
the final decision made by humans.
On the consumer front, several
virtual support tools would increase
and become mainstream. It will be
almost expected to come across these
bots first before talking to any human
at all. However, only businesses
that demonstrate a customer-centric
approach would thrive in these
scenarios, while others would struggle
to adapt to the right technology. And,
most importantly, “What do you want
to do when you grow up?” will soon
become an obsolete question.
AI will change the job market
entirely as there will be growing
requirements for soft skills, since
most hard skills will be automated.
Especially for the Indian economy,
since we have mostly relied on hard
skills for local as well as global
opportunities, this will pose a
significant challenge to keep up with
declining demands. We will be forced
to come up with new business models,
not just as businesses but also as
an economy.
Maintaining a balanced approach
Regardless of how the recent or long-
term future with AI looks like, there are
a few points that we must understand
and accept in entirety. Most of these
points align with OECD’s AI principles
that were released in early 2019.
AI systems should benefit humans,
the overall ecosystem on the planet, and
the planet itself by driving inclusive
growth, sustainable development and
well-being of all. These systems must
always be designed such that they
respect and follow the rule of law
and rights of the ecosystem (humans,
animals, etc). These should also respect
the general human value system and the
diversity it exhibits. More importantly,
there must be appropriate safeguards
in the system such that humans are
always in the loop when necessary,
or can intervene if they feel the need,
regardless of necessity. After all, a fair
and just society should be the goal of
any advancement.
Creators of AI systems should
always demonstrate transparency
By: Anand Tamboli
The author is a serial entrepreneur, speaker, award-winning published author and
emerging technology thought leader.
The article was originally published in the February 2020 issue of Electronics For You.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  25
Insight For U & Me
and responsible disclosure about the
functionality and methodology of the
system. People involved and affected by
this need to understand how outcomes
are derived and, if required, should be
able to challenge them.
Any AI system should not cause
harm to users or general living beings,
and must always function in a robust,
secure and safe way throughout its
lifecycle. Creators and managers of
these systems have the responsibility to
assess continually and manage any risks
in this regard.
Most importantly, on the
accountability front, anyone creating,
developing, deploying, operating
or managing AI systems must
always be held accountable for the
system’s functioning and outcomes.
Accountability can drive positive
behaviours and thereby potentially
ensure that all the above general
principles have been adhered to.
There is a general feeling that
over-regulation limits innovation and
advancements. However, there is no
point in racing to be the first; instead,
let us strive to be better. Being fast and
first by compromising on ethics and
quality is certainly not an acceptable
approach by any means.
It is unlikely that in the next
ten years or so, we will have robots
controlling humans. However,
technology consuming us, our time,
feelings and mindfulness is very much
a reality even today; and it is getting
worse day by day already.
Just one wrong turn in this fast lane
is what it will take to cause regression
for society. The rise of AI should not
lead to the fall of humanity. Let us work
towards keeping the technology, AI or
otherwise, in our control, always!
For U & Me Insight

Breaking Down the Buzz

Around Quantum Computing
Quantum computing helps scientists and researchers in solving problems above a certain
complexity. Quantum computers derive their power by utilising quantum mechanics and
marvels such as superposition and entanglement, which allows them to perform a variety
of computational tasks exponentially faster than classical computers.

T
he invention of the computer
has undeniably been one of
the biggest technological
revolutions in the history of
mankind. However, classical computing
is not the only way that was formulated
in the last century to solve complex
problems. While it was in 1927 that
physicist Heisenberg introduced the
uncertainty principle, it was not until
1970 that using quantum mechanics
as a communication resource and the
term quantum information theory
came into being. So, the question
arises: how is quantum computing
different from classical computing,
and how did it go from being a purely
theoretical subject to being used by
companies like Google and IBM?
Quantum computers versus
classical computers
We already have computers and even
supercomputers for faster processing
speeds, then why do we need quantum
computers? To understand this we
need to understand the difference
between the two.
A classical computer’s main
purpose is to save and manipulate
data for working. Its chip uses bits to
store this information. These bits are
like tiny switches with two states—on
and off, represented by one and zero,
respectively. From every pixel in an
image to the texts exchanged between
people, everything is ultimately made
up of these bits, a language that the
computer understands. But even
supercomputers cannot define the
uncertain state that exists between
on and off, especially on atomic and

(Credit: www.eweek.com)

26  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


molecular levels. This makes them
capable of only analysing simple
molecules in practical applications
related to biology and chemistry.
This is where scientists and
researchers needed to find a
better way of computing when
the probability is involved (such
as spinning a coin instead of
flipping it). Also, for problems
above a certain complexity, more
computational power is required,
which is only possible with
quantum computers.
Dr Martin Laforest, senior
product manager and quantum
technology expert, Isara,
explains, “Quantum computers
leverage the surprising and often
counterintuitive behaviour of
atoms and molecules, making them
radically different from today’s
computer. They derive their power
by utilising quantum mechanics
and marvels such as superposition
and entanglement. Their quantum
behaviour makes them much
more powerful, allowing them to
perform a variety of computational
tasks exponentially faster than IBM’s quantum computer model
classical computers.”
Behind the magic: Working of a
quantum computer
Quantum computers use quantum bits
(qubits) instead of the regular bits. By
combining qubits, a lot more data can
be processed in less time as compared
to basic computers. Underlying
quantum computing is the principle
of quantum mechanics. Fundamental
quantum properties like superposition,
entanglement, and interference are used
to manipulate the state of a qubit.
Superposition refers to the
overlapping of usually independent
states. Real-life examples include
sounds generated while playing an
instrument where notes are played
simultaneously, or waves formed on
throwing a stone in a lake. Qubits can
be in superposition, that is, somewhere
between on and off. This means that
if there is more than one option, a
quantum computer can go through each
option simultaneously and choose the
ultimate answer, instead of ruling out
previous options individually before
checking the next option.
Entanglement is a quantum
phenomenon where states of two
particles, even if they are physically
separated, are tied such that they
cannot be described independently
of each other. Whatever is the result
of measuring one of these particles,
the outcome of another one will
be mathematically related to it.
This correlation is necessary for
faster computations through special
instructions (algorithms) that can only
be written with quantum computers.
Just like wave interference, quantum
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  27
Insight For U & Me
states can also change (cancel or add)
depending on whether they are in
phase or out of phase.
Making of a quantum computer
According to IBM researcher
DiVincenzo’s Criteria, there are five
minimal requirements for creating a
quantum computer—a well-defined
scalable qubit array; an ability to
initialise the state of the qubits to a
simple fiducial state; a universal set of
quantum gates; long coherence times,
much longer than the gate-operation
time; and single-qubit measurement.
There are different ways
to create a qubit. In one of the
most commonly used methods,
superconductivity is applied
to create and preserve hard-to-
maintain quantum states. Quantum
computers are isolated from any
sort of electrical interference and
made to operate in an environment
at almost absolute zero temperatures
to prevent errors while working
with qubits. Superconductors
also minimise energy loss during
transmission.
In order to achieve results close
to absolute zero, necessary cooling
power is made available and multiple
other steps are followed. This includes
attenuation during refrigeration to
protect qubits from thermal noise
during the process of transmitting
signals to the quantum processor. Also,
cryoperm shields protect the qubits
from electromagnetic radiation.
To create a fault-tolerant quantum
system, it is necessary to increase the
computational power of a quantum
computer. For this, higher numbers
of qubits are preferable as states
increase exponentially with each qubit.
Researchers have designed algorithms
for sequential quantum operations
that can run on fault-tolerant quantum
computers for extended periods of
time. To ensure that the results are
accurate and noise-free, low error rates
need to be maintained.
For U & Me Insight
New opportunities
Quantum computers are not just about
saving time and money with better
speed and higher efficiency of doing
tasks that can already be performed.
Quantum computers are thought to be
useful in places where any uncertain
system needs to be simulated.
Quantum chemistry is one of the
most promising applications of quantum
computing. By determining the lowest
energy state among various molecular
bond lengths that represents equilibrium
molecular configuration, it is possible
to simulate a molecule. Modeling even
simple molecules and predicting particle
interactions in chemical reactions can
aid in discovery of new life-saving
medicines and other compounds useful
for making efficient devices, which is not
possible with conventional computing
memory and processing power.
Another application is cryptography,
as these computers can easily generate
hard-to-break encryption keys for
better cyber security. Dr Laforest says,
“Quantum computing promises many
positive disruptions. One such possibility
is the use of quantum particles called
photons to create secure communications
channels for the distribution of
quantum keys. This has the potential
to revolutionise networking and the
protection of future data transmission.”
Other possibilities include improved
solar panels, financial strategies for
prediction of financial markets, better
weather forecasts, and so on.
Problems encountered
Holding an object in a superposition state
for a long period is difficult. Interaction
with the environment is necessary for
quantum measurement. But if a qubit
comes in contact with such occurrences
as changing magnetic and electric
fields and radiation from warm objects
nearby, or if there is a cross talk between
qubits, it undergoes decoherence, and
changes in the uncertain state cause
the information to be lost. Due to these
interference problems, in spite of high
30  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
speeds, a quantum computer is also much
more vulnerable to errors than a classical
computer would be.
This requires robust quantum
processes and designing devices such
that the system is only sensitive to
the targeted measurement, protecting
quantum states from decoherence. But
one needs to keep in mind that high
sensitivity is crucial for precision.
When it comes to cybersecurity, Dr
Laforest says, “It is important to note that
quantum is a dual-use technology that
also has the capacity to cause security
chaos. Imagine a world where our
existing encryption is no longer effective.
If hackers equipped with quantum
computers can break these algorithms,
they can determine the private keys
used to secure the data and expose it.
The combination of quantum computer
speed-ups and known algorithms
developed by Peter Shor and Lov Grover
make this possible. To ensure we are
ready, preparation has to start now as it
will take a decade or more to fix many
of our complex systems. Experts in the
field of quantum computer development
agree it is highly probable we are
only ten years away from large-scale
quantum computing with this capability.
Cybersecurity experts are already
starting to prepare for potential quantum
computer attacks on the encryption
algorithms we use to protect data today.”
“The good news is that quantum-safe
algorithms exist. The most significant
challenge we face is the development
of tools and methods that will make
it easy and seamless to transition to
these quantum-safe algorithms. Key
to this success will be successfully
completing this transition within a
decade without any wholesale disruption
to our current security systems and
infrastructure,” Dr Laforest adds.
Dearth of required skill set, lack
of available resources, and cost are
some other impediments for it to be the
technology in the making.
The article was originally published in the May 2020 issue of Electronics For You.
Can it be a staple technology?
Companies are competing to build
reliable quantum computers in a variety
of fields like manufacturing, security
and financial services. Numerous
startups like Isara, Ionq and 1qbit have
come up in this field. They are willing
to invest money in the technology at
this early stage of development because
of its great potential. Cloud based
quantum computing technology is
increasingly being leveraged to make it
more easily available for remote access
and be user-friendly for enterprises,
no matter the size of their work
teams. Microsoft in November 2019
announced that it would start providing
access to quantum computers in its
Azure cloud for select customers.
IBM Quantum designed and built
the world’s first integrated quantum
computing system ‘IBM Q System
One’ for commercial use in 2019.
Another company, D-Wave, recently
announced that it is freely opening
up its quantum computers to anyone
who has ideas for how to use them
to find a cure for COVID-19. For
assistance in developing solutions, the
company along with its customers like
Cineca, Volkswagen, Denso, Tohoku
University, Kyocera, Sigma-i, among
others, is offering access to their
engineering teams.
Quantum computers could change
the world but they are still not advanced
enough to replace the classical method.
They are not so useful when it comes
to the basic tasks like storing images
that ordinary computers do. They
are undoubtedly powerful, but not
so reliable yet. As for now, the most
beneficial way will be to give users
access to both traditional and quantum
computers simultaneously.
By: Ayushee Sharma
The author is a technology journalist
at EFY.

Introduction to Green
Computing and its Importance
Foundation of green computing was laid as far back as 1992 with the launch of
the Energy Star program in the USA. The success of Energy Star motivated other
countries to take up the subject for investigation and implementation.
A
ny technology that aspires
to be nature-friendly ought
to be green. Recognition
of this fact has led to
development of green generators,
green automobiles, green energy, green
chemistry as well as green computing.
Green computing is a leap forward
for information technology (IT), and
more specifically for information and
communication technology (ICT).
Green computing has emerged as the
next wave of ICT.
Motivation for the subject of
green computing arose to protect the
environment against hazards generated
at three different states of ICT, namely,
information collection (by electronic
devices), information processing
(through algorithms and storage) and
information transportation (through
networking and communication).
Carbon dioxide accounts for about
eighty per cent of global warming. As a
rule of thumb, if world-wide increasing
application of ICT is assumed to
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  31
Overview For U & Me
contribute at least twenty per cent
towards carbon dioxide, ICT becomes
responsible for sixteen per cent of
global warming. This is undoubtedly a
cause of concern. As per one research-
based estimate, fifty billion devices
like computers, mobile phones,
sensors, actuators and robots shall
connect to the Internet by this year’s
end, creating even more havoc.
Of course different strategies would
be needed to nudge ICT towards green
computing, which is necessary to reduce
the pollutants generated in collection,
processing and transportation of the
information. In today’s scenario,
primary challenge in achieving green
computing is to realise energy-efficient
devices, energy-efficient processing
and energy-efficient networking.
Invariably, energy efficiency is
required to address reduction in heat
dissipation that is basically responsible
for emission of carbon dioxide.
In case of electrical, electronic or
computer systems, wasteful heat is
generated due to thermal vibration of
particles in the components. Therefore
any green initiative should have direct
or indirect motivation to reduce this
thermal vibration. Reduced circuitry or
a minimal system helps in reducing the
number of vibrating particles.
Minimal circuit designs, which
lead to technologies of very large
scale integration (VLSI) or ultra large
scale integration (ULSI), are now
well-established technical solutions.
These solutions meet the objectives
of realising low cost and smaller-size
systems. It was never thought these
For U & Me Overview
E-waste generation by region 2014 (metric tonnes/year)
20
15
10
5
0 whereas in conventional computers
Oceania Africa
Figure 1: Increasing e-waste trend world over (Credit: UNU - The Global E-Waste Monitor 2014)
would also indirectly provide a solution
for reducing particles in vibration.
In the process of minimisation, two
more revolutionary technologies have
emerged: molecular scale of electronics
(MSE) and quantum computing. It
was the quest for an ever decreasing
size, but more complex electronic
components with high speed ability,
which gave rise to MSE.
The concept that molecules may be
designed to operate as a self-contained
device was put forwarded by Forrest
L. Carter. He proposed some molecular
components analogous of conventional
electronic switches, gates and
connections. Accordingly, the idea of a
molecular P-N junction emerged. MSE
is a simple interpolation of IC scaling.
Scaling is an attractive technology.
Scaling of FET and MOS transistors
is more rigorous and well defined
than that of bipolar transistors. But
there are problems in scaling of
silicon technology. In scaling, while
propagation delay should be minimum
and packing density should be high,
these should not be at the expense of
the power dissipated. With these scaling
rules in minds, scaling technology of
silicon is reaching a limit.
Dr Barker reported that, “change,
spin, conformation, colour, reactivity
and lock-and-key recognition are just a
few examples of molecular properties,
which might be useful for representing
and transforming logical information.
32  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
Europe Americas Asia
To be useful, molecular scale logic will
have to function close to the information
theoretical limit of one bit on one carrier.
Experimental practicalities suggest
that it will be too easy to construct
regular molecular arrays, preferably by
chemical and physical self-organisation.
This suggests that the natural logic
architectures should be cellular automata:
regular arrays of locally connected finite
state machines where the state of each
molecule might be represented by colour
or by conformation. Schemes such as
spectral hole burning already exist for
storing and retrieving information in
molecular arrays using light. The general
problem of interfacing to a molecular
system remains problematic. Molecular
structures may be the first to take practical
advantages of novel logic concepts such
as emergent computation and ‘floating
architecture’ in which computation is
viewed as a self-organising process in a
fluid-like medium.”
Change is the only thing that
is permanent in the universe. In
technology scenario, changes become
inevitable means of evolution and
revolution. In tune, a new generation
of IT known as Quantum Computing
(QC) has come up. Mechanical
computing, electronic computing,
quantum computing, DNA computing,
cloud computing, chemical
computing and bio computing are a
few generation-wise migrations of
information technology (IT).
In the conventional computers
we work with now, computing and
processing of data is based on transistors’
on and off states as binary representation
of ‘1’ or ‘0’ or vice versa. In quantum
computers, the basic principle is to use
quantum properties to represent data.
Here, computation and processing of
data is made with quantum mechanical
phenomena such as superposition,
parallelism and entanglement. Therefore,
data is represented by binary ‘bits,’ in
quantum computers representation is
done with ‘qubits’ (quantum bits).
Qubits are typically subatomic
particles such as electrons and photons.
Generating, processing and managing
qubits is an engineering challenge.
Superiority of quantum computing over
classical computing is multi-fold. First,
whereas in classical computing logical
bits are represented by on and off states
of transistors, in quantum computing
qubits are harnessed by properties of
subatomic particles. Size of quantum
computers will thus be much smaller
than that of present-day computers.
Both MSE and QC are thus found to be
indirect solutions for green computing.
At current state of technology
march, green ICT may be better looked
at as a challenge to realise eco-friendly
and environmentally-responsible
solutions in order to reduce just not
heat dissipation but also to maximise
energy efficiency, recyclability and
bio-degradability. Fact is, fast-growing
production of electrical, electronic and
computing equipment has resulted in
enormous increase of e-waste, and
especially carbon dioxide, which
is responsible for creating havoc in
the environment and for increasing
pollution. As per a report published by
International Telecommunication Union
(ITU), e-waste has increased rapidly
and reached a global high. Increasing
trend of e-waste all over the world is
shown in Figure 1.
Many studies have established that
computers and IT industries dissipate

Total CO2 emissions 2017 (gigatons)
Figure 2: Global carbon dioxide emissions (Credit: Wikipedia.org)
more energy than others. Impact of
ICT industries on emission of carbon-
dioxide is immense. As shown in
Figure 2, India is currently the third
largest producer of carbon dioxide.
Urgent solutions required at the
level of hardware design management
include minimal configuration, adoptive
configuration, consolidation by
virtualisation, algorithmic efficiency,
optimal resource utilisation, optimal
data centres, optimal link utilisation,
limiting power by reducing cable
Overview For U & Me
2017 CO2 emissions per person (tons/person) of e-waste, air management and
cooling management, among others.
For ICT scientists and engineers, the
challenge will be to design technology
and algorithms to minimise particle
vibration, travel path and heat loss
due to input-output mismatch. Design,
operational and transmission related
thermal loss are core issues of ICT.
This makes production of green ICT
a great challenge, although, as parts
of its implementation, energy-smart
devices, sleep-mode devices, cluster
computing, cloud computing, etc are
already in place.
Foundation of green ICT was laid
as far back as 1992 with the launch of
the Energy Star program in the USA.
The success of Energy Star motivated
other countries to take up the subject
for investigation and implementation.
Leading countries working on green
ICT now include Japan, Australia,
Canada and the European Union.
Formalisation of green ICT is in fact
due to standards proposed by IEEE
who has formalised Green Ethernet and
802.3az-enabled devices for green ICT.
Green ICT is a clean-environment-
based technology. However, fruitful
realisation of green ICT is equally
dependent upon awareness in society.
Society needs to practice common
ethics of ‘don’t keep computer on, when
not needed,’ ‘don’t use Internet as a free
tool, but as a valuable tool of necessity
only,’ ‘don’t unnecessarily replace
devices after devices just because
you can afford to’ and so on. Without
length, minimising protocol overhead, societal responsibility, technology alone
protocol for compressed header, cannot ensure achieving the objectives
green networking, management of green ICT.
By: Prof. Chandan Tilak Bhunia, Abhinandan Bhunia
Prof. Chandan Tilak Bhunia, PhD in computer engineering from Jadavpur University,
is fellow of Computer Society of India, Institution of Electronics & Telecommunication
Engineers, and Institution of Engineers (India).
Abhinandan Bhunia did B S in computer engineering from Drexel University, USA and
MBA from University of Washington, USA.
The article was originally published in the April 2020 issue of Electronics For You.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  33
For U & Me Overview
How Technology is Helping
Fight Coronavirus
Latest technologies like artificial intelligence,
blockchain, chatbots, face recognition, robots, drones
and software solutions are all contributing to the
fight against the fast-spreading coronavirus that has
caused havoc all over the world.
I
n the battle against novel coronavirus (Covid-19), emerging
technologies have stood out by making immense
contribution in an unexpected, creative and amazingly
responsive way. Delivery drones, disinfecting robots,
smart helmets, and thermal imaging cameras are all being
deployed in the fight against Covid-19. Latest technologies
are being used to predict and combat spread of the infectious
disease. These technologies include artificial intelligence (AI),
analytics software, chatbots, apps, telemedicine, blockchain,
and advanced facial recognition software.
The better we can track the virus, the better we can fight it.
Advanced AI has been used to help diagnose the disease and
accelerate the development of a vaccine. Google’s DeepMind
division has used its latest AI algorithms and computing power
to understand the proteins that might make up the virus, and has
published their findings to speed up the process of treatments.
Several drug companies are also using AI-powered drug
discovery platforms to search for possible treatments.
AI-based systems are being used to detect coronavirus
infection via CT scans with 96 per cent accuracy. Portable lab-
on-chip detection kits are helping medical teams on the ground
to identify infected individuals for proper medical care quickly.
These tools are helping remote areas with limited medical
resources to immediately screen out suspected coronavirus-
infected patients for further diagnosis and treatment.
Blockchain-powered services are helping hospitals to
spend less time on administrative work and allocate staff to the
frontlines. Blockchain platforms speed up claims processing
and minimise the need for face-to-face contact amidst the
coronavirus outbreak.
Chatbots are being used to share information and free
online health consultation services. These can answer queries
related to the virus such as symptoms, preventive measures
and treatment procedures.
Software solutions that are transforming the healthcare
industry include hospital management, mobile healthcare,
telemedicine, and wearables. Telemedicine enables remote
monitoring and care for patients. It can provide necessary
34  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
services for people infected with Covid-19. Mobile healthcare
solutions empower healthcare providers and patients by
creating a platform for interaction and medical care services.
Face recognition technology is being used in surveillance
systems that can recognise people, even while they are wearing
masks, with a relatively high degree of accuracy. The surveillance
systems use facial recognition technology and temperature
detection software to identify people who might have fever or
are not wearing masks. Facial recognition technology has been
integrated with thermal imaging to make fever-detection cameras.
Contactless temperature detection software, AI-powered non-
contact infrared sensor systems, and smart helmets that can measure
the temperature of anyone within five-metre radius can quickly
detect a person who is suspected of having a fever. These are being
deployed at stations, airports, schools, malls, community centres
and other public places that have large gatherings.
Robots are being used to clean or sterilize hospitals,
and perform basic diagnostic functions, to minimise the
risk of cross-infection. The robots allow physicians to
communicate with the patient via a screen, and are equipped
with a stethoscope to help doctors take a person’s vitals while
minimising exposure to the staff. They can deliver food and
medicine to reduce the amount of human-to-human contact.
Robots use ultraviolet light to autonomously kill bacteria and
viruses in quarantine wards without human intervention.
Drones are being used for contactless medicine delivery,
and for spraying disinfectant around the country, especially
in quarantine zones. These transport medical samples and
conduct thermal imaging. Drones are also deployed to
check travellers’ temperatures and disposal of hospitals’
medical waste.
By: Deepshikha Shukla
The author is a freelance technology journalist.
The article was originally published in the April 2020 issue
of Electronics For You.
 Overview For U & Me

Role of Technology
in Maintaining Law and Order
With criminals becoming tech-savvy, police and the courts also need to know the
latest tools and make use of the latest technologies like cyber policing, artificial
intelligence, data analytics, blockchain and cloud computing.

R
educing crime rates and
delivering speedy justice to
the needy is a challenging
task globally. It requires
smart policing and effectively dealing
with roadblocks in legal proceedings.
Although the adoption is slower
when compared to other sectors like
finance and insurance due to the
ethics constraints involved, science
and technology are being increasingly
leveraged by enterprises to accomplish
this mission. This is proved by the
success of Global Legal Hackathon 2019
that was organised to develop technical
solutions for the industry, in which
participation of over 6,000 people across
46 cities in 24 countries was seen.
As criminals become more tech-
savvy, police personnel need to know
the latest tools and scientific methods
to keep up with them. Automating
manual activities that are rule-based
and repetitive through robotic process
automation can save time and manpower.
Investigating and understanding the
intricacies of even the most pressing
cases has become easier with the aid
of CCTV footage, which also serves as
valuable evidence in the court, especially
in the absence of witnesses. Digitisation
of cases ensures full electronic access to
(Credit: www.tagesspiegel.de)
past history of all records. Analysing the
data from first-hand evidence at the site
in real time provides a good overview of
cases from the beginning.
Telangana Police is among those
taking diverse initiatives since past
few years to maintain law and order
in the state. For instance, mobile
apps like TSCOP, ePetty case, Cop
Connect, Facial Recognition System,
and e-Challan system launched by the
department have played a huge role in
data sharing and structuring the system.
Continue to page 39...

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  35

Focus
Why Devops is Popular
Six Things to Consider for a DevOps
Transformation to the Cloud
The COVID-19 reality has
pushed the market towards
faster adoption of remote
access to IT by developers.
Software vendors are
therefore in a race to enable
and expand cloud DevOps solutions. Increasingly, teams
seek to adopt end-to-end DevOps platforms or tool bundles
that decrease their reliance on multiple vendors and give
them ownership of the tooling infrastructure. But what
should an enterprise demand from cloud DevOps tooling,
and what key differentiators should be considered?
The Five Best DevOps Tools
DevOps emerged out of the
agile software development
movement and applies
some similar standards to
the application life cycle
management (ALM) process. DevOps is hard to characterise
since it’s to a greater degree a development or logic than
an unbending arrangement of standards or practices.
Open source DevOps tools are used to streamline software
improvement and arrangement. Here’s a brief description of
the five best tools amongst these.
DevOps is the Future of Software
Development DevOps scores over legacy,
monolithic and agile
software development.
This article discusses
the various stages of
the DevOps software
development cycle and
delineates the appropriate
FOSS tools that can be
used at each of the stages.
Gulp: A DevOps Based Tool
for Web Applications
This article highlights the
features of Gulp, an open source
cross-platform streaming
task runner that lets software
developers automate many
development tasks. It covers the
installation of Gulp, and touches
upon the code required for task
and module management.
How DevOps Differs from
Traditional IT and Why
DevOps is the buzzword in
the software development
industry. But how much of
the hype associated with this
new technology is warranted?
This article demystifies
DevOps and explains to the
reader why it scores over
traditional IT practices.
Understanding DevOps:
A Revolution in Software
Development
In the field of IT and software
development, DevOps
implementation is gaining
significant popularity. This
article takes a quick look at the
important concepts of DevOps
and the phases of its life cycle.
It also maps the relevant
open source tools, and finally,
highlights how it can bring
value to the IT industry.
 Focus

Six Things to Consider

for a DevOps Transformation
to the Cloud
The COVID-19 reality has pushed the market towards faster adoption of remote
access to IT by developers. Software vendors are therefore in a race to enable and
expand cloud DevOps solutions. Increasingly, teams seek to adopt end-to-end DevOps
platforms or tool bundles that decrease their reliance on multiple vendors and give
them ownership of the tooling infrastructure. But what should an enterprise demand
from cloud DevOps tooling, and what key differentiators should be considered?

H
ere are six things one
should consider when
engaging with any
individual vendor for
a DevOps digital transformation
to the cloud.
End-to-end (E2E) solutions
Developers today are looking for an
E2E solution and an all-in-one user
experience. However this doesn’t mean
that they will compromise on the ‘best
of breed’ approach. Therefore, DevOps
platform providers should offer a Class
A tool stack as part of their platform,
in addition to very strong ecosystem
integrations and plugins to make
developers’ lives easier, respecting the
freedom of choice they want.
An E2E platform also requires a
vendor commitment to allow a true
‘one-browser solution’ and not simply
bundled tools that are integrated.
This ensures that the user will have a
full experience from a single UI that
connects all services.
Universal package management
All of the metadata and dependencies
in your myriad technologies must
be supported (such as Docker, npm,
Maven, PyPi, Golang, NuGet, Conan,
etc – but also the 20+ more that
you may find in your portfolios).
Point solutions for single or limited
technology types will only serve to
frustrate your development teams
and require the adoption of multiple
solutions and repositories within your
organisation. Large enterprises have

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  37

Focus

Figure 1: Comparing cloud based end-to-end DevOps solutions

not only myriad technologies, but also
a long legacy of deployed, mission-
critical applications that must be
supported at scale with local, remote
and virtual repositories.
Both cloud and on-premise
solutions need to be integrated
100 per cent
This isn’t about whether or not to have
a cloud solution. Many companies
that offer cloud solutions don’t have
corresponding on-premise/self-hosted
options, or vice-versa. More still
have completely separate solutions
that provide different features and
methods that don’t talk to each other,
requiring you to learn a new product,
user experience and user interface. As
you transition to a cloud environment,
both cloud and on-premise solutions
need to be able to function in the same
way 100 per cent of the time to ensure
a smooth transition. For instance, as
you go through a cloud migration, you
will need the same tools and functions
in both places in order to keep the
business running.
Multi-cloud
While you might think one cloud is
enough, you should select a vendor that
provides services across and between
all major clouds. Keep your options
open and your peace of mind intact
by avoiding any vendor lock-in and
providing maximum resilience.
Security
Security is an integrated part of
the pipeline that supports all of
your package types and it is now
a line-item for many companies.

38  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


Cloud DevSecOps tools should
make it possible to block artifact
downloads (or break builds) that
contain vulnerabilities, requiring
tight integration all the way into the
repository. Security policies should be
easy to define and manage across your
repositories. And any cloud security
solution should allow you to easily
identify the impact of a vulnerability
across the entire DevOps pipeline.
The world of containers also comes
with a challenge. Your DevSecOps
tools should be able to ‘open’ any
container and scan several tiers in, and
with all packages look for dependencies
that include vulnerabilities.
A DevOps platform should always
strive to be ahead of any hacker and
Continue from page 35...
Another such attempt comes from
IIIT Delhi, where a research centre
has been built to assist the capital’s
police for such purposes as criminals’
identification, cyber policing, traffic
management, and combating crimes
by using artificial intelligence (AI),
biometrics, image processing, Big
Data, social media analysis and
network forensics.
Technologies like AI, analytics,
blockchain, and cloud computing are
making their way into the courtrooms,
too. AI-powered tools can be used
by lawyers for most daily tasks, from
reviewing documents and performing
legal research rapidly to predicting
various outcomes of a case. Businesses
can utilise AI to review contracts for
partnerships without any bias, and
perform background checks before
hiring new employees to avoid getting
into legal troubles later.
Several companies in countries like
the US, Singapore, the UK, Canada
and Australia are using technologies
to solve issues in this space.
Canada-based Kira Systems uses
machine learning (ML) for contract
secure all software packages in the
pipeline from build-to-production.
Cloud-ready CI/CD
Traditionally, application
development teams were responsible
for creating localised CI/CD
(continuous integration/continuous
delivery) automation. This approach
provides short-term gains for the
individual teams but ends up being
a constraint in the long run since
enterprises get no economies of scale
across their CI/CD implementations.
A modern CI/CD provider
should support and scale enterprise-
wide workflows (aka the ‘software
supply chain’) that span all popular
technologies and architectures
analysis. UK’s Tessian employs AI to
secure confidential data and emails
for law firms.
In India, researchers from IIT
Kharagpur have recently developed
an AI-powered method to automate
the reading of legal case judgements,
case law analysis and enhance legal
search across several domains. Deep
neural models enable understanding
the rhetorical roles of sentences or
jargons in a judgement when adequate
data is available, and hence aid in
organising legal documents.
Blockchain finds application in
email encryption, verifying processes
and securing evidence, like in financial
transactions and many other purposes.
For example, Legaler’s blockchain and
developer tools provide infrastructure
to build decentralised applications for
legal services. In 2017, Global Legal
Blockchain Consortium was formed to
drive the standardisation of blockchain
technology in the legal sector. It has
already surpassed 150-member mark,
The article was originally published in the April 2020 issue of Electronics For You.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  39
Focus
of today as well as keep pace with
technology evolution. It should
provide a way to assemble pipelines
from pre-packaged building blocks,
rather than developing them from
scratch. These pipelines can be
templatised and shared as libraries
across the organisation, thereby
building a knowledge base that is
constantly growing and improving.
In other words, your CI/CD provider
should give you economies of scale
over time in the cloud, and help you
ship code faster.
By: Jens Eckels
The author is the director, product
marketing at JFrog.
which includes law firms, software
companies, and universities, among
others.
According to a 2019 report titled
LawTech Adoption Research by tech
analysts TechMarketView, the number
of lawtech companies has grown over
the past few years, but the adoption
rate is not that high among legal
practitioners. One of the major reasons
noted behind this is the partnership
model in which spending is done from
the partners’ profit pool.
To change this scenario, the US
states like North Carolina and Florida
have already mandated technology
training for CLE (continuing legal
education) credits. Pressure from clients
for cheaper offerings is also pushing
law firms to move to cloud computing
and other tech solutions.
By: Ayushee Sharma
The author is a technology journalist
at EFY.
Focus

The Five Best DevOps Tools

DevOps emerged out of the agile software development movement and applies
some similar standards to the application life cycle management (ALM) process.
DevOps is hard to characterise since it’s to a greater degree a development or logic
than an unbending arrangement of standards or practices. Open source DevOps
tools are used to streamline software improvement and arrangement. Here’s a brief
description of the five best tools amongst these.

D
evOps, which began by
uniting engineers and tasks,
has now turned out to be a
key tool in the most basic
parts of the software development
life cycle. With the introduction of
cloud computing and virtualisation,
the requirement for new systems
DEVELOP TEST
administration processes has increased.
The DevOps mantra is, “Automate
and monitor the procedure of software
creation, extending from integration,
testing and releasing, to deploying and
overseeing it.”
Stages of the DevOps life cycle
The following are the five phases of the
DevOps life cycle, and the popular tools
used in each phase.
1. Continuous integration – Jenkins
2. Configuration management –
Ansible, Chef and Puppet
3. Continuous inspection – Selenium
4. Containerisation – Kubernetes
5. Virtualisation – Parasoft Virtualize
Now, let’s list the top five tools
among these.
Ansible
This open source tool monitors
application deployment, configuration
management, orchestration and so on.
The Ansible development steps are
given in Figure 1.
Key features
ƒƒ It has an agentless design.
ƒƒ It is powerful due to the work

MONITOR DEPLOY

Figure 1: Ansible development steps (Image source:

https://www.ansible.com/integrations/) Figure 2: Chef Code

40  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


Figure 3: Client server architecture of Docker
process arrangement.
ƒƒ I t is straightforward and
simple to use.
Chef
This tool is used for checking the
software designs. Figure 2 shows how
to create, test and deploy Chef code.
Key features
ƒƒ It guarantees that your configuration
strategies will stay adaptable,
versionable, testable and
intelligible.
ƒƒ It helps to normalise configurations.
ƒƒ It automates the entire procedure of
guaranteeing that all frameworks
are accurately designed.
Docker
Docker uses the idea of containers
that virtualise the operating system.
It very well may be used to bundle
the application (for instance, the
WAR document) alongside the
conditions to be utilised for sending
in various situations. The client server
architecture of Docker empowers
the customer to associate with the
daemon, which plays out the tasks like
structure, running and distributing the
compartments (Figure 3).
Key features
ƒƒ Docker’s compactness is made
possible due to its unique
Figure 5: Old vs new ways of Kubernetes
innovations in containerisation,
frequently found in
independent units.
ƒƒ It bundles everything that
an application requires to
run — libraries, framework
apparatuses, runtime, and
so on.
Puppet Enterprise
This is an open source
configuration management
tool that designers and activity
groups can use to safely work
on programs (infrastructure,
applications, etc) at any place. It
empowers clients to comprehend
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  41
Focus
Node
2
1 Catalog
3 Facts
Report
Puppet Master
Figure 4: Working flow of Puppet
and follow up on the progressions
that occur in applications, alongside
the inside and out reports and real-
time alerts. Clients can distinguish
those changes, and remedy the issues.
Please refer to Figure 4.
Key features
ƒƒ It can work on hybrid infrastructure
and applications.
ƒƒ It has the client-server architecture.
ƒƒ It bolsters the Windows, Linux and
UNIX working frameworks.
Kubernetes
Kubernetes, often abbreviated to K8S,
is a container orchestration tool that
Focus

Tools Pros Cons

Ansible 1. SSH-based, so it doesn’t require the installation of 1. Less powerful than tools based in other programming languages.
any agents on remote nodes.
2. Variable registration is required for even basic functionality, which can make
2. Playbook structure is simple and clearly structured. easier tasks more complicated.
Chef 1. Extendable with scripting language, easy 1. Web UI does not allow modification of configurations
configuration files

2. Easy to implement, learn and use on daily basis for 2. Other platforms have better pre-configured deployment scripts
configuration management
Docker 1. Docker produces an API for container management 1. Containers don’t work at bare-metal rates.

2. Fast application deployment 2. Data storage is intricate

Puppet 1. Well-established support community through 1. For more advanced tasks, you will need to use the CLI (Command Line
Puppet Labs. Interface), which is Ruby based.
2. Because of the DSL (Domain Specific Language) and a design that does not
2. Simple installation and initial setup focus on simplicity, the Puppet code base can grow large and unwieldy.

Kubernetes 1. Microservices rolling updates 1. Businesses need a certain degree of reorganization when using Kubernetes
with an existing app.
2. Makes it a lot easier to establish effective CI/CD 2. Pods sometime need a manual start/restart before they start
pipelines. working as intended. This can happen in certain situations such as
when running near full capacity

Figure 6: Pros and cons of the five tools featured

ƒƒ ontainer grouping using Pod
C
Ansible Chef Docker Kubernetes Puppet
ƒƒ Self-healing
Configuration Configuration It’s a Configuration Configuration ƒƒ Auto-scalability
management management container management tool. management tool.
tool. tool. technology. ƒƒ DNS management
It is written in Ruby and Written in Go Written in Go Ruby-DSL (domain- ƒƒ Load balancing
Python. Erlang. programming programming language. specific language) ƒƒ Rolling updates or rollbacks
language. language.
ƒƒ Resource monitoring and logging
Easy for Complex Easiest to Setting up Kubernetes Difficult for Figure 6 depicts the pros and cons
configuration from the manage, requires a lot of planning beginners
management development understand of the five tools featured here. Figure
perspective and isolate. 7 compares these five top DevOps
It is more It’s similar to It is very Kubernetes is suitable It’s more targeted automation tools.
appropriate Puppet different for developers of modern towards operations
for front-end from the applications that don’t require
developers, others and development The road ahead
where some has several background
programming components
The DevOps universe is brimming
might be with unique and remarkable open
needed.
source tools. Mainstream DevOps
It is similar Builds a It delivers Kubernetes brings a lot It may configure tools can help you overcome any
to Puppet in pipeline of configuration of complexity as it is a more than one
producing files. processes. for one larger project with more process at a time hindrance. You can select the device
process at a moving parts that any and make the that suits your business needs and
time, making other DevOps tool, but dependencies a bit
Docker files that also leads to latency complex. immediately watch the improvements
simpler than in executing commands, unfold in your business activities.
bash script making troubleshooting
for process and monitoring Diverse DevOps tools will not only
configuration. cumbersome. work well independently, but also
Figure 7: Comparing the top five DevOps tools work well in combination.

takes containerisation to the next level. containers at scale. With Kubernetes,

It functions admirably with Docker or you can gather your containers into
By: Dr S. Balakrishnan
any of its options. Kubernetes is still legitimate units.
The author is a professor at Sri Krishna
exceptionally new; its first version College of Engineering and Technology,
turned up in 2015. It was established Key features Coimbatore. He has 17 years of
by a few Google engineers who were Some of the platform features that experience in teaching, research and
administration.
looking for a solution to oversee Kubernetes offers are:

42  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Focus

Understanding Continuous
Integration and Continuous
Delivery/Deployment
This article discusses continuous integration (CI) and continuous delivery/deployment (CD),
which are part and parcel of the DevOps software development culture. The goal of all
developers is to produce software that is reliable, reusable, extendable, flexible, correct
and efficient. DevOps ensures this, with CI and CD as integral parts of the process. 

I
n simple terms, integrate when
you commit. Continuous
integration implementation
doesn’t mean less bugs. Rather, it
Development
highlights the issues or bugs in the early Code Commit
stages and hence is useful because the
earlier in the development cycle you
fail, the faster you recover!
Static Code
Analysis (SCA)
The benefits of detecting
bugs early
In the case of discovering failures or
bugs, it becomes the priority of the SCA - Quality
respective stakeholders to focus on Gate
solving build or continuous integration
issues at the earliest and fix the
broken build.
Continuous integration (CI) is a Compilation
popular DevOps practice that requires
the development team to commit code
into a shared repository (Centralised
Version Control or Distributed Version Unit Test
Control) as and when a feature is Execution
completed or a bug is fixed. Each
commit goes through a build validation
process using an automated build
Code Coverage
process with any automation tool -Quality Gate
feasible, based on the knowledge or the
culture of the organisation.
It is important for the development
team to commit frequently whenever a Build Package
feature is implemented or a bug is fixed.
There are still some developers who
commit code even if it is not properly
tested or is not working fine. There are
also instances when code is committed Figure 1: Continuous integration

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  43

Focus
Figure 2: Benefits of continuous integration
Figure 3: Best practices of continuous integration
44  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
even if the compilation fails; or when
libraries are hard coded and the path
in the developer’s system is different,
and hence the compilation fails.
The solution
Please don’t commit if you are not
ready. This piece of advice holds good
for life as well as while developing
software! After each commit, build
validation has to pass, all quality gates
have to be cleared and things must
work successfully in the pipeline.
Even if the pipeline fails, it must
be fixed immediately. This failure
helps the Jenkins engineer and
developer to grow.
Failure should be the motive to
inspire us to continue to fix problems
and improve continuously even if a
temporary failure creates a road block.
We must not avoid build failures but
address them head on. Build success
is only one step away when the
issues are fixed.
When you decide to go down the
path of cultural transformation, think
about the satisfaction you’ll get when
after addressing multiple failures, your
software eventually gets the green status.
Benefits, best practices and
challenges of CI
We can understand some of the benefits
of continuous integration from Figure 2.
Best practices make it easy to
implement continuous integration
within an organisation, without
resistance. Please refer to Figure 3.
The major challenge with continuous
integration is the mindset! People resist
it because all developers are so used to
the IDE, its features and functionality
that any new tool is not welcomed. It
is important to start with the proof-of-
concept and a demo of all the automation
activities orchestrated in the automation
server. This helps to develop a mindset
that is ready to accept new practices.
Once the resistance is overcome, the
development team can use CI practices
effectively because it helps to boost
productivity.
Continuous delivery
Continuous delivery and continuous
deployment are the next logical steps
when implementing DevOps practices.
Continuous integration creates an
artifact that needs to be deployed in a
development or QA stage, or a production
environment. The next step is automated
deployment in different environments.
These environments can be on-premise or
on the cloud. Resources can be physical
or virtual machines, or containers.
Services can be Infrastructure as a Service
or a Platform as a Service, where the
artifact can be deployed.
The continuous delivery phase
represents activities to deploy an
artifact in non-production environments
using scripts or deployment tools. The
artifact is always ready to be deployed
in production. Usually, continuous
delivery is a preferred practice since
there’s less risk involved with it.

Figure 4: CI/CD
Figure 5: Benefits of continuous delivery
The continuous deployment phase
represents activities to deploy the
artifact in production environments
using scripts or deployment tools.
The artifact is directly deployed
in production. Usually, continuous
delivery is preferred to continuous
deployment.
Benefits, best practices and
challenges of continuous
delivery
Success and failure are both equally
important during digital and cultural
transformation activities. Let’s
understand some of the benefits of
continuous delivery from Figure 5.
Focus
Figure 6: Best practices of continuous delivery
The best practices to create code for an application or
that make it easy to different components of an application.
implement continuous Most of the time, environments are
delivery are shown in controlled by customers and they are
Figure 6. not willing to relinquish this due to
A major challenge security reasons; hence, automated
with continuous deployment to all the environments
integration is gaining control over the might not be feasible all the time.
environment and people’s mindset. Cloud resources and containers
The dev environment might be in are changing the game. The way cloud
control of the development team, yet services are used for infrastructure as
there are specific instances when this well as for the DevOps setup helps
is not the case. VDI (virtual desktop to automate deployment in different
infrastructure) is given to developers environments.
References
[1] Agile, DevOps and Cloud Computing with Microsoft Azure: https://www.amazon.in/
Agile-DevOps-Cloud-Computing-Microsoft/dp/9388511905
[2] Continuous Integration: https://www.martinfowler.com/articles/
continuousIntegration.html
By: Mitesh S.
The author has written the book, ‘Agile, DevOps and Cloud Computing with Microsoft
Azure’. He contributes occasionally to https://clean-clouds.com and https://
etutorialsworld.com.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  45
Focus

Building the DevOps Pipeline

with Jenkins
Jenkins is an open source tool that provides integration with the tools used in
application life cycle management to automate the entire process, based on feasibility.

T he prerequisites for installing Jenkins on a specific

system are:
ƒƒ Jenkins official documentation recommends
java -jar jenkins.war --httpPort=9999

Access Jenkins’ dashboard: Browse to http://

256MB of RAM.
ƒƒ Jenkins official documentation recommends 10GB
of drive space; however, I suggest 50GB - 80GB of
free space on a disk drive.
ƒƒ Jenkins official documentation recommends
Java 8 or 11.
The commands to run Jenkins are as follows.
You can start Jenkins at the command line by using
the Generic Java package (.war):
java -jar jenkins.war
To access the Jenkins dashboard, browse to http://
localhost:8080. Change the port to run Jenkins, with the
following command:
localhost:9999
It will redirect the user to unlock the Jenkins page
by providing the administrator password, which can be
found from the console log when the jenkins command is
executed for the first time.
Click on Install Suggested Plugins.
Once all the plugins are installed, create your first
admin user (Figure 2) and click on Save and Continue.
Provide the Jenkins URL or keep the default,
depending on your needs.
Once the Jenkins setup is complete, click on Start
using Jenkins. Verify the Jenkins dashboard. Click on
Manage Jenkins.
Go to Manage Jenkins -> Global Tool Configuration.

Figure 1: Customise plugins

46  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Focus

Figure 5: Global Tool Configuration

Figure 2: Admin user

Getting Started

Instance Configuration
Jenkins URL: http//localhost:9999/

The Jenkins URL is used to provide the root URL for absolute links to various
Jenking resources. That mean this value is required for proper operation of many
Jenkins features including email notification, PR status updates, and the BUILD_
URL environment variable provided to build steps.

The proposed default value shown is not saved yet and is generated from the
current request, if possible. The best practice is to set this value to the URL that
users are expected to use. This will avoid confusion when sharing or viewing links. Figure 6: Upstream and downstream projects

Figure 3: Jenkins instance configuration application life cycle management activities. There are two
types of pipelines in Jenkins, as of today. This means that
Jenkinsfile can contain two different types of styles/syntax
and yet it can achieve the same thing.
Scripted pipelines follow the imperative programming
model. They are written in the Groovy script in Jenkins. All
Groovy blocks/constructs help to manage flow as well as
error reporting.

node {
/* Stages and Steps */
}

Figure 4: Jenkins dashboard node {

stage(‘SCA’) {
The Build Pipeline plugin // steps
Using Jenkins and the build pipeline makes us realise the }
complexities of managing Jenkins over time. It is easier to use stage(‘CI’) {
the Build Pipeline plugin in Jenkins to create pipelines with // steps
upstream and downstream projects (Figure 6). }
Execute the pipeline as shown in Figure 7. stage(‘CD’) {
// steps
Scripted pipelines }
Jenkins 2.0 and later versions have built-in support for }
delivery pipelines. Jenkinsfile contains the script to automate

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  47

Focus

Figure 7: Build Pipeline plugin

Figure 8: Blue Ocean plugin

pipeline {
Declarative pipelines agent any
As the name suggests, a declarative pipeline follows a stages {
declarative programming model. Declarative pipelines are stage(‘SCA’) {
written in a domain-specific language in Jenkins that is clear steps {
and easy to understand. //

48  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Focus

}
}
stage(‘CI’) {
steps {
//
}
}
stage(‘CD’) {
steps {
//
}
}
}
}

Blue Ocean
Blue Ocean provides
an easy way to create
a declarative pipeline
using new user
experiences available
in its dashboard. It is Figure 9: Blue Ocean repository
like creating a script by
selecting components,
steps or tasks.
Open the Blue Ocean
dashboard and click on
Create Pipeline. Connect
with the required
repository; then create
stages, select the steps
and configure.
Blue Ocean is a new
user experience, and it Figure 10: Blue Ocean tests
provides an easy way to
access unit test results
(Figure 10).
Click on Pipeline
to get the status of the
pipeline. Click on the
specific stages to access
the logs of these stages.
Click on Artifacts
to access the package
file and other artifacts
(Figure 12).
The following list of
open source tools can be
integrated in the pipeline
to implement DevOps
practices. Figure 11: Blue Ocean automated deployment

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  49

Focus

Figure 12: Blue Ocean artifacts

Tool Description

Travis CI Hosted continuous integration service that supports integration with BitBucket and GitHub.
https://travis-ci.org/

GoCD GoCD is a build and release tool that helps to perform end-to-end orchestration for application life
cycle management activities.
https://www.gocd.org/

Nagios Nagios is an open source tool that can be used to monitor network and infrastructure.
https://www.nagios.org/

Docker This is a very popular container management tool. Kubernetes supports Docker as a container pro-
vider.
https://www.docker.com/

Ansible Ansible is used for automation, such as for configuration management and continuous delivery.
https://www.ansible.com/

Collectl This is used to gather the performance data of systems such as CPU, network, and data.
http://collectl.sourceforge.net/

GitHub This provides a repository for public and private access to maintain version control. It is hosted.
https://github.com/

Kubernetes This is one of the most popular container orchestration tools available in the market.
https://kubernetes.io/

Artifactory This provides community and enterprise versions of artifact management tools.
http://www.jfrog.com/artifactory/

CruiseControl CruiseControl is a Java based, open source continuous integration framework.

http://cruisecontrol.sourceforge.net/

50  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Focus

Selenium Selenium is a popular automated functional testing tool that is used for Web applications. It is open
source.
https://www.selenium.dev/
Appium Appium is a popular automated functional testing tool that is used for mobile applications.
It is open source.
http://appium.io/
SonarQube This is used to analyse the code to track bugs, security vulnerabilities, and code smells. It supports
more than 15 programming languages.
https://www.sonarqube.org/
SaltStack https://www.saltstack.com/
Apache JMeter https://jmeter.apache.org/
OWASP ZAP This is used to scan security issues of applications for penetration testing. It is an active open source
Web application security scanner.
https://www.zaproxy.org/
Ant Apache Ant is an XML based build management tool for Java based projects.
https://ant.apache.org/
Gradle This is a popular build management tool for Android based projects. It is also used in Java based ap-
plications. It supports domain-specific languages.
https://gradle.org/
Maven Apache Maven is one of the most popular build tools, with multiple goals for an application’s life cycle
phases such as build, test, and deploy. It is mainly used for Java projects.
http://maven.apache.org/
Hygieia This is a one-of-a-kind DevOps dashboard that helps to integrate with tools such as Bamboo,
Jenkins, Jenkins-codequality, Jenkins Cucumber, Sonar, AWS, uDeploy, XLDeploy, Jira, VersionOne,
Gitlab, Rally, Chat Ops, Score, Bitbucket, GitHub, Gitlab, Subversion, GitHub GraphQL, HP Service
Manager (HPSM), AppDynamics, Nexus IQ and Artifactory. It has two types of dashboards — one for
engineers and the other for executives.
https://www.capitalone.com/tech/solutions/hygieia/
CFEngine This is a popular DevOps tool that is used to automate IT infrastructure related operations.
https://cfengine.com/
GitLab This is a Git repository. Now it also provides support for automation pipelines to configure continuous
integration and continuous deployment.
https://about.gitlab.com/
Junit This is a popular yet simple unit testing framework to write unit tests for the Java programming lan-
guage.
https://junit.org/
Jasmine Jasmine is a popular behavioural data driven framework to verify JavaScript based applications.
http://jasmine.github.io/

References
[1] Agile, DevOps and Cloud Computing with Microsoft Azure.
https://www.amazon.in/Agile-DevOps-Cloud-Computing- By: Mitesh S
Microsoft/dp/9388511905
[2] Continuous Integration. https://www.martinfowler.com/ The author has written the book ‘Agile, DevOps and Cloud
articles/continuousIntegration.html Computing with Microsoft Azure’. The title of his upcoming
book is ‘Hands-on Azure DevOps’.

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  51

Focus

Understanding DevOps:
A Revolution in Software
Development
In the field of IT and software development, DevOps implementation is gaining
significant popularity. This article takes a quick look at the important concepts of
DevOps and the phases of its life cycle. It also maps the relevant open source tools,
and finally, highlights how it can bring value to the IT industry.

T
he word DevOps is DevOps is gaining popularity Various stages in the
a combination of DevOps is gaining popularity DevOps life cycle
development and because it bridges the gap between An understanding of DevOps
operations. DevOps is developers and the supporting is incomplete without having
not software — not a tool, not a operations team. The DevOps knowledge of its life cycle. The
product and not a programming approach aims at: various phases that comprise the
language, but an approach whereby ƒƒ Delivering high-quality software DevOps life cycle are highlighted in
the development and operations in a shorter development life Figure 1 and described below.
teams work together, instead of cycle. 1. Continuous development: The
waiting for the other to finish tasks, ƒƒ Deployment in frequent cycles. first phase of the DevOps life
to enable continuous delivery of ƒƒ Reducing the time to move cycle involves ‘planning’ and
value to end users. This software into production, from ‘coding’ of the software. Planning
development methodology improves conceptualising an idea. includes activities like designing
the collaboration between the ƒƒ Accelerating application delivery the blueprint of the module, and
development and the operations across enterprise portfolios. identifying the resources and the
teams using various automation tools. ƒƒ Enabling rapid building and algorithm to be used. Once the
These tools are implemented during delivery of products. plan is finalised, the developers
different phases, which are a part of ƒƒ Delivering applications code the application and maintain
the DevOps life cycle. and services at high velocity. it using popular tools like Git,
Gradle and Maven.
2. Continuous testing: To
PLAN catch any error and ensure
Continuous DEPLOY
Development Continuous
the reliability of the software,
CODE Deployment the developed modules are
OPERATE
continuously tested for bugs.
BUILD
Integration In this phase, testing tools like
Continuous Testing Selenium (Se), TestNG, JUnit,
Continuous
TEST
MONITOR Monitoring etc, are used.
3. Continuous integration:
This phase is the heart of the
Continuous Integration
DevOps life cycle in which code
supporting new functionality in
Figure 1: Different phases of the DevOps life cycle the Git repository is continuously

52  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


PLAN
DEVELOPER
CODE
BUILD
TEST
Figure 2: Categories of popular DevOps tools
reflected using the Jenkins tool.
4. Continuous deployment:
The deployed applications
are consistently improved by
collecting end user experiences
from the obtained results of the
software. Popular tools that are
used for the deployment stage are
Ansible, Puppet and Docker.
5. Continuous monitoring:
This is the stage when the
performance of the deployed
product is monitored. In this
phase, the production server is
continuously monitored for all
kinds of activities that happen
on it, and the log files are
generated. Nagios is a popular
tool for this phase.
Careful implementation of these
five phases of the DevOps life cycle
helps an IT organisation to achieve
its business objectives. Various tools
that contribute to the DevOps life
cycle are highlighted in Figure 2.
The pros and cons of
adopting DevOps
The pros and cons that should be
considered when adopting DevOps
are listed here.
Focus
DEPLOY
OPERATIONS
OPERATE
Continuous
Integration
MONITOR
The pros The cons
ƒƒ It facilitates faster code ƒƒ It takes time for users to adjust to
deployment into production. the frequent changes that are made.
ƒƒ Ensures the release of ƒƒ Lack of a separate plan for consistent
quality products. upgradation may lead to severe
ƒƒ It helps in reducing the chances shortfalls with respect to security.
of software failures. ƒƒ IT companies need to hire
ƒƒ Enables continuous delivery of experts who have an in-depth
software with stable features. understanding of development,
ƒƒ Helps in identifying bugs at testing, monitoring and deployment
an early stage of development tools, and that can be a challenge.
due to better communication With technology evolving very
between the development and rapidly, IT companies that do not
operations teams. adopt DevOps may find it difficult to
ƒƒ Implementation of DevOps also compete in the market. The adoption
increases the organisation’s of the DevOps model is inevitable,
opportunities for innovation because it bridges the gap between the
and research. developer and operations teams.
References
These links are in addition to those provided in the body of the article itself.
[1] https://devops.com/
[2] https://www.edureka.co/blog/devops-lifecycle/
[3] https://intellipaat.com/blog/tutorial/devops-tutorial/
[4] https://www.simplilearn.com/cloud-computing/devops
[5] A. Bhardwaj and C. Rama Krishna, ‘A Container-based Technique to Improve Virtual
Machine Migration in Cloud Computing’, IETE Journal of Research, pp. 1-16, 2019.
By: Dr Aditya Bhardwaj
The author works as assistant professor at PEC (a deemed university), Chandigarh. He
has experience in cloud computing and Big Data technology.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  53
Focus

How DevOps Differs from

Traditional IT and Why
DevOps is the buzzword in the software development industry. But how much of the
hype associated with this new technology is warranted? This article demystifies DevOps
and explains to the reader why it scores over traditional IT practices.

CODE DEPLOY

PLAN RELEASE

BUILD OPERATE

DEVELOPMENT TEAM OPERATION TEAM

TEST MONITOR

D
evOps is a set of practices
that promotes partnership
between the development
and operations teams to
deploy code to production faster in an
atomically and continuous manner. The
word ‘DevOps’ is a combination of the
words ‘development’ and ‘operations’.
DevOps helps to increase an
organisation’s speed to provide
applications and services. It allows
companies to serve their customers
better and compete more powerfully in
the market. In simple words, DevOps
can be interpreted as an arrangement
of development and IT operations with
better communication and collaboration.
Why it is needed
ƒƒ B
efore DevOps, the development
and operations teams worked in a
completely segregated way.
ƒƒ Testing and deployment were
different activities done separately
after design-build. Hence, they
required more time.
ƒƒ Using traditional IT practices, team
members spent large amounts of
their time in testing, deploying and
designing, instead of building the
project.
ƒƒ Manually deploying the code leads
to human errors in production.
ƒƒ Different teams have different
timelines. This leads to problems
of synchronising timelines, causing
more delays in deployment.
Comparing DevOps and
traditional IT
When comparing traditional IT ops with
DevOps, it’s clear how they differ and
why the latter is increasingly embraced
by organisations worldwide. Given
below are some points of comparison.
1. Time
DevOps teams spend 33 per cent more
time to refine infrastructure against
failure than traditional IT ops teams. In
addition, DevOps teams spend about 21
per cent less time putting out fires on a

54  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


weekly basis and 37 per cent less time
handling support cases. DevOps teams
also waste less time on administrative
support due to a higher level of
automation, self-service tools and
scripts for support tasks. With all of
this additional time, DevOps teams are
able to spend 33 per cent more time to
ameliorate infrastructure and 15 per
cent more time on self-enhancement
through further education and training.
2. Data and speed
DevOps teams tend to be small, agile,
driven by innovation and focused on
addressing tasks in an accelerated
manner. According to a Gartner
report, they work on the mantra,
“Don’t fail fast in production; embed
monitoring earlier in your DevOps
cycle.” Agility is one of the top five
objectives of DevOps. In the case of
traditional IT ops, the data count for
the feedback loop is confined to the
service or application being worked
upon. If there’s a downstream effect
that is not known or noticed, it can’t
be addressed. It’s up to IT ops to
pick up the pieces. That is the reason
why DevOps is faster in delivering
business applications and the
challenge to IT ops is to keep pace
with the speed of business.
3. Recovery and crunch time
The average DevOps teams see only
two app failures per month, and
the recovery time is less than 30
minutes for over 50 per cent of all
respondents. Of the DevOps teams
surveyed, 71 per cent can recover
from failures in less than 60 minutes
while 40 per cent of traditional IT ops
teams need over an hour to recover.
A key practice of DevOps is to be
prepared for the possibility of failures.
Continuous testing, alerts, monitoring
and feedback loops are put in place
so that DevOps teams can react
quickly and effectively. Traditional
IT ops teams are almost twice as
likely to require more than 60 minutes
to recover, while recoveries in less
than 30 minutes are 33 per cent more
likely for DevOps teams. Automated
deployments and an infrastructure
that’s programmable are the key
features for quick recovery.
4. Release of software
When it comes to releasing software,
DevOps teams need roughly 36.6
minutes to release an application
whereas traditional IT ops teams
need about 85.1 minutes. This means
that DevOps teams release apps
more than twice as fast as traditional
IT ops teams.
Why DevOps is better
There are many advantages of using
DevOps rather than traditional IT.
ƒƒ Reduced chances of product
failure: Software delivered by
DevOps teams is usually more
fit-for-purpose and relevant to
the market thanks to the constant
feedback loop.
ƒƒ Improved flexibility and support:
Applications developed by
DevOps teams are typically more
expansive and easy to maintain
due to the use of microservices
and cloud technologies (we’ll get
to that later).
ƒƒ Faster time to market: Application
deployment becomes quick and
dependable thanks to the advanced
continuous integration (CI) and
automation tools DevOps teams
usually count on.
ƒƒ Better team efficiency: DevOps
means joint responsibility, which
leads to better team engagement and
productivity.
ƒƒ Clear product vision within the
team: Product knowledge is no
longer spread across different roles
and departments, which means
better process transparency and
decision making.
The DevOps culture comes with
a variety of rewards, some of which
include greater efficiency, security and
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  55
Focus
organisational collaboration. The 2017
State of DevOps Report quantifies
this increase in efficiency, reporting
that high performing organisations
employing DevOps practices spend 21
per cent less time on unplanned work
and rework, and 44 per cent more
time on new work. More generally
speaking, however, successfully
implementing DevOps practices can
have a profound impact on a company
through improving efficiency and
execution in areas that are both
essential and decidedly unglamorous.
Fredrik Håård, an engineer
with over 12 years of DevOps
experience who worked as a senior
cloud architect at McKinsey and
at Wondersign, articulates this
point more fully, “Good DevOps
engineers must be champions – and
take responsibility for all the areas
that might not be prioritised by the
organisation such as data security,
disaster recovery, mitigation, and
audits.” He adds, “The choices you
make in DevOps can have long-lasting
effects at a company.”
So the conclusion is, DevOps
teams get more time and solve
problems faster. They spend more
time in ameliorating things, less time
fixing things, recover from failures
faster and release applications more
than twice as fast as traditional IT
ops. Through DevOps, all members
of different teams work together
because they have the same goal,
which is to deliver quality software
to the market.
References
[1] https://en.wikipedia.org++++
[2] https://www.toptal.com
By: Neetesh Mehrotra
The author works at NIIT Technologies
as a senior test engineer. His areas
of interest are Java development and
automation testing.
Focus

RCloud is DevOps for

R
Data Science
DevOps is the collaboration between the development and deployment streams of a
software system. It increases productivity by reducing the time between the development
and deployment of software. RCloud is a platform that accelerates data analysis related
insights by reducing the time between coding and deployment.

D
evOps for data science is
gaining popularity as it
involves infrastructure,
configuration management,
integration, testing and monitoring.
Hence, it accelerates data analysis
insights. DevOps supports data
scientists by creating integrated
environments for most vital tasks such
as data exploration and visualisation.
Data scientists need varied types of
infrastructure to handle any complex
project. DevOps does the provisioning
and configuration of infrastructure for
a variety of environments. Any data
science model is iterative in nature
as it involves new data that needs
to be trained, and based on that, it
evolves new models that need to be
made available to users. For this,
the data scientist applies continuous
integration and deployment practices.
DevOps bridges the gaps between the
training environment and the model
deployment environment through
continuous integration and continuous
deployment pipelines.
RCloud is considered as DevOps
for data science as it resolves the data
analysis development and deployment
issues of collaboration, sharing,
scalability and reproducibility. RCloud
was created at AT&T labs by Simon
Urbanek, Gordon Woodhull and Carlos
Scheidegger and is open source software.
It is a Web based platform for all aspects
of data science such as analytics,
visualisation and collaboration. It uses
the R language for all tasks.
Features of RCloud
ƒƒ C an be used from anywhere: It is
browser based software; so if you
have Internet connectivity, you can
use it from anywhere.
ƒƒ Project container facility: RCloud
notebook consists of all the
required components and associated
data dependencies. It contains
dependencies of data analysis that
include code, comments, equations,
visualisations, etc.
ƒƒ Association feature: RCloud
provides an excellent capability
for association. It is browser based
software that is installed on a server
or any distributed environment
such as Hadoop. It provides access
to all notebooks so that new users
can view, copy, edit and update the
analysis and visualisations with new
data sets. It is very efficient and easy
to use as it only needs a browser.
ƒƒ Distribution capability: RCloud is
based on the URL sharing concept.
It delivers value faster. It is agile
enough for us to share the URL at
any stage of analysis.
ƒƒ Scalability: RCloud can perform
parallel connections to multi-
server systems. It provides the
data scientist with flexibility to
run Big Data packages without
writing complex code.
ƒƒ User directory based access:
It contains a user directory that
provides access to the notebooks of
every user registered with RCloud.
ƒƒ Reproducibility: A data analysis
on RCloud can be verified and
executed by anyone with access to
the notebooks without any concern
for environmental variables.
ƒƒ Live code execution: RCloud
notebooks are not static Web pages
but code that is executed live.
ƒƒ Unique RCloud Web service
interface: RCloud provides a
unique Web service interface
through which any notebook
asset can be integrated with other
technologies by simple means.
ƒƒ Promotes user engagement:
RCloud is platform-independent.
Access and control remains
constant, which increases user
confidence and engagement.
Why RCloud?
For effective results of a data science
project, certain information must
be shared among team mates. They
must be agile enough to address new
features and functionalities. They must
move their results at different levels
of the data science project such as
data pre-processing, exploratory data
analysis, predictive modelling and
visualisation. RCloud is the perfect
software to address these issues as
it contains association, distribution,
scaling and reproducibility (ADSR)
characteristics. RCloud is very vital
for data science for many reasons:
ƒƒ RCloud is an open source Web
based platform for data science. It
has excellent capability to help you
share your ideas or work with your

56  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


Figure 1: RCloud on GitHub
team mates. Figure 1 describes the
RCloud components on GitHub.
ƒƒ RCloud provides a platform for the
data scientist to search relevant things
without reinventing the wheel.
ƒƒ It provides fast interaction with
data in the Hadoop Distributed File
System (HDFS) or similar kinds of
distributed file systems. This feature
is very well suited to Big Data
analytics.
ƒƒ RCloud differs from other DevOps
of data science as it provides
browser based access.
ƒƒ It gives a lot of flexibility to users to
create any type of complex widgets,
notebooks or dashboards.
ƒƒ Both registered and non-registered
users can view or interact with
live notebooks of the RCloud
environment.
ƒƒ Communication in RCloud is done
with standard communication
protocols such as HTTP.
ƒƒ RCloud provides data scientists
with the capability to run Big
Focus
Data packages without writing ƒƒ R Cloud maintains automatic Git
complex code. based trials of code modifications.
ƒƒ RCloud is well suited for Big Data DevOps involves infrastructure,
applications due to its scalability configuration management,
feature. integration, testing and monitoring.
ƒƒ It provides great security features RCloud is considered as DevOps for
so that unauthorised clients cannot data science as it resolves the data
make calls to the RCloud runtime analysis development to deployment
environment. Notebooks of RCloud issues of ADSR. In this article, we
can also be encrypted for advanced have described the main features of
security. Authenticated client server RCloud in detail, with an emphasis
channelling is also a unique feature on those that are very essential for
of RCloud. data scientists.
References
[1] https://www.forbes.com/sites/janakirammsv/2018/11/04/the-growing-
significance-of-devops-for-data-science/#7120d32a7481
[2] http://stats.research.att.com/RCloud/
[3] https://www.kdnuggets.com/2016/11/rcloud-devops-data-science.html
By: Dr Dharmendra Patel and Dr Atul Patel
Both the authors are associated with the Smt. Chandaben Mohanbhai Patel Institute of
Computer Applications, Charusat, Gujarat. Their areas of interest are data mining, data
science, artificial intelligence, deep learning and image processing.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  57
Focus

How Prometheus Helps to

Monitor a Kubernetes
Deployment
The ubiquitous usage of Kubernetes necessitates accurate and timely monitoring
of its clusters. Prometheus and Sensu are the tools to latch on to for this.

D
evelopment combined
with operations leads to
the high impact DevOps
practices, and microservices
based architecture is ubiquitous in
such an environment. Although such
architecture is not new — it’s been
around since the 1980s – the DevOps
practice is relatively new. The idea
of DevOps began in 2008 with a
discussion between Patrick Debois
and Andrew Clay Shafer, concerning
the concept of agile infrastructure. In
June 2009, the epochal presentation
of ‘10+ Deploys a Day: Dev and Ops
Cooperation’ was made at Flickr by
John Allspaw and Paul Hammond.
That year can be treated as the year
the DevOps movement began. The
Linux containerisation technology
based Docker came into play in
2013. In 2014, we saw the journey of
Kubernetes begin as an orchestration
attempt of multiple Docker containers
from Google. Subsequently, Kubernetes
began to be maintained by the Cloud
Native Computing Foundation
(CNCF). The consequent proliferation
of Kubernetes led to the absolute
necessity of monitoring its deployment
in DevOps. This resulted in open source
tools like Prometheus and Sensu, along
with the older Nagios. Let’s take a
closer look at Prometheus, while getting
the basics on Sensu.
Monitoring tools
We typically talk about the following
three tools in the context of Kubernetes
monitoring.
1. Prometheus: This open source
Kubernetes monitoring tool
collects the metrics of Kubernetes
deployment from various
components, relying on the pull

58  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


method. The data is stored in an
inbuilt time series database.
2. Sensu: This complements
Prometheus. It provides flexibility
and scalability to measure the
telemetry and execute service checks.
The collected data can be more
contextual here, and it is possible
to extend it to provide automatic
remediation workflows as well.
3. Nagios: This is the old friend we
used to monitor deployments.
It serves its purpose well,
especially in the context of bare
metal deployments to a host of
administrators.
Kubernetes monitoring
challenges
When we decide to go with
Kubernetes, we are ready to embrace
the mantra that the only constant in
life is change. Naturally, this adds
associated challenges with respect to
monitoring Kubernetes deployments.
Let’s discuss a few monitoring
challenges to understand them better.
Short-lived
jobs
push metrics
at exit
Pushgateway
pull
metrics
Node
Jobs/
exporters
Prometheus
targets
Figure 1: Prometheus architecture (Reference: https://prometheus.io/docs/introduction/overview/#architecture)
1. The apps are ever evolving; they
are always moving.
2. There are many moving pieces to
monitor. Kubernetes is not really a
monolithic architectural tool, as we
all know. Those components also
keep on changing.
3. Once upon a time, everything was
server based, often bare metal. Then
we got cloud based deployment. The
natural progression makes multi-
cloud a reality, which adds an extra
facet to the monitoring challenges.
4. We typically annotate and tag
the pods and containers in a
Kubernetes deployment to give
them nicknames. The monitoring
should reflect the same.
Kubernetes data sources
Once we understand the challenges
of monitoring in a Kubernetes
environment, we need to understand
the moving parts of a typical, sizable
deployment. These moving parts are the
data sources from which Prometheus
pulls the monitoring metrices.
Service discovery
file_sd
Alertmanager
discover
targets
Prometheus server
HTTP
Retrieval TSDB
server
PromQL
HDD/SSD
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  59
Focus
1. The hosts (nodes) which are
running the Kubelets.
2. The process associated with
Kubelets — the Kubelet metrices.
3. Kubelet’s build in the cAdvisor
data source.
4. The Kubernetes cluster. This is
the whole deployment. From a
monitoring perspective, these are
Kube state metrices.
The good part is that Prometheus
can monitor all of the above, addressing
all the monitoring challenges we
described earlier.
The Prometheus architecture
The heart of Prometheus is its server
component, which has a collector
that pulls the data from different data
sources to store it in an inbuilt time
series database (TSDB). It also has an
API server (HTTP server) to serve the
API request/response.
The unique feature of Prometheus
is that it is designed for pulling out data
to make it scalable. So, in general, one
need not install an agent to the various
Prometheus
alerting
Email
notify
etc
push
alerts
Prometheus
web UI
Grafana
Data
visualisation
and export
API clients
Focus
Figure 2: The Prometheus target dashboard
Figure 3: The Prometheus graph dashboard
components to monitor them. However,
Prometheus also has an optional push
mechanism called Pushgateway.
The Alert Manager is the
component to configure the alerts and
send notifications to various configured
notifiers like email, etc. The PromQL is
the query engine to support the query
language. Typically, it is used with
Graphana, although Prometheus has an
intuitive inbuild Web GUI. Data can be
exported as well as visualised. For the
sake of brevity, we are only covering
the Prometheus Web GUI, not the
Graphana based visualisation.
60  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
So let us play around with To configure Prometheus.yml, use
Prometheus. the following code:
It is now time to get our hands
dirty. A typical Prometheus installation global:
involves the following best practices: scrape_interval: 30s
1. Install kube-prometheus. evaluation_interval: 30s
2. Annotate the applications with
Prometheus’s instrumentation. scrape_configs:
3. Label the applications for easy - job_name: ‘prometheus’
correlation. static_configs:
4. Configure the Alert Manager for - targets: [‘127.0.0.1:9090’,
receiving timely and precious alerts. ‘127.0.0.1:9100’]
Let us configure Prometheus in a labels:
demo-like minimalistic deployment to group: ‘prometheus’
get a taste of it.

To start Prometheus, type:
docker run -d --net=host \
> -v /root/prometheus.yml:/etc/
prometheus/prometheus.yml \
> --name prometheus-server \
> prom/prometheus
For starting the node exporter, the
code snippet is:
docker run -d \
> -v “/proc:/host/proc” \
> -v “/sys:/host/sys” \
> -v “/:/rootfs” \
> --net=”host” \
> --name=prometheus \
> quay.io/prometheus/node-
exporter:v0.13.0 \
> -collector.procfs /host/proc \>
-collector.sysfs /host/sys \
> -collector.filesystem.ignored-
mount-points “^/(sys|proc|dev|host|etc)
($|/)”
Note: The paths shown here are
purely local to a demo deployment.
You may have to adjust the paths as
per your own deployment.
1. Raw metrices can be viewed by
curl localhost:9100/metrices. In the
URL, it can be viewed in /metrics
endpoint.
2. The metrics view can be found
in /targets URL. For production,
Graphana is the preferred tool.
3. The graph view can be found in
/graph URL. Any metrics collected
(viewable using /metric endpoint)
can be viewed in /graph.
Sensu: Complementing
Prometheus
Prometheus has lots of advantages
because of its scalability, pluggability
and resilience. It is tightly integrated
with Kubernetes and has a thriving
supportive community. However,
it has a few disadvantages also. For
example, the simplified constrained
Focus
Sensu
Container Container Agent
(Sidecar)
Pod Sensu Agent
Pod
(DaemonSet)
Kubelet
Sensu Agent (Kubelet)
Pod
Node
Figure 4: Sensu deployment (Reference: https://blog.sensu.io/monitoring-kubernetes-
docker-part-3-sensu-prometheus)
data model loses various contexts of the is of paramount interest for self-
measurements. It is tightly integrated healing purposes.
with Kubernetes, so if the deployment 3. Secure transport: Prometheus
is multi-generational, it may face does not encrypt or authenticate
challenges. The data transported is the metrics.
neither authenticated nor encrypted. In this article, we discussed how
Sensu complements Prometheus on microservices based architectural
this aspect aptly. Figure 4 shows how patterns are getting contextual in
Sensu agents are typically deployed in a a DevOps environment with the
Kubernetes environment. ubiquitous usage of Kubernetes.
Some of the advantages Sensu Monitoring of such a sizable
provides to complement Prometheus deployment becomes challenging
are as follows: because of its highly dynamic
1. Adds more context to Prometheus nature. So, we need tools like
collected data to make it more Prometheus to overcome some of
meaningful. the monitoring challenges. Sensu
2. Workflow automation of acts as a complementary tool to
remediation actions based on alerts. Prometheus, improving the latter’s
So, alerts become actionable. This performance considerably.
References
[1] https://dzone.com/articles/devops-tools-for-monitoring
[2] https://blog.sensu.io/monitoring-kubernetes-part-1-the-challenges-data-sources
[3] https://blog.sensu.io/monitoring-kubernetes-docker-part-2-prometheus
[4] https://prometheus.io/docs/introduction/overview/
[5] https://blog.sensu.io/monitoring-kubernetes-docker-part-3-sensu-prometheus
[6] https://www.katacoda.com/courses/observability-analysis/prometheus
By: Pradip Mukhopadhyay
The author has 19 years of experience across the stack — from low level systems
programming to high level GUIs. He is a FOSS enthusiast and currently works for
NetApp, Bengaluru.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  61
Focus

DevOps vs Agile:
What You Should Know
About Both
DevOps is a practice of bringing the development and operations teams
together, whereas agile is an iterative approach that focuses on collaboration,
customer feedback and small rapid releases. This article highlights the
differences between the two software development technologies.

D
evOps is a software
development method that
focuses on communication,
integration and collaboration
among IT professionals to enable rapid
deployment of products. On the other
hand, the agile methodology involves
continuous iteration of development
and testing in the SDLC process. In
this software development method, the
emphasis is on iterative, incremental and
evolutionary development.
Key differences
ƒƒ D
evOps is a practice of bringing
the development and operations
teams together, whereas agile is an
iterative approach that focuses on
collaboration, customer feedback
and small rapid releases.
ƒƒ D evOps focuses on constant testing
and delivery while the agile process
focuses on constant changes.
ƒƒ DevOps requires a relatively large
team while agile requires a small
team.
ƒƒ DevOps leverages both shift-left
and shift-right principles; on the
other hand, agile leverages the shift-
left principle.

Customer Developer Operations

+ + +
Software Requirement Tester IT Infrastructure

Figure 1: The communication chain between stakeholders in a typical IT process

Customer Operations
Developer
+ +
+
Software Requirement IT Infrastructure
Tester

Figure 2: Agile addresses the communication gaps between the customer and developer

62  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Focus

Customer Developer Operations

+ + +
Software Requirement Tester IT Infrastructure

Figure 3: DevOps addresses the communication gaps between the developer and IT operations teams

Differences between agile and DevOps

Parameter Agile DevOps

What is it? Agile refers to an iterative approach that fo- DevOps is considered a practice of bring-
cuses on collaboration, customer feedback and ing the development and operations teams
small, rapid releases. together.

Purpose Agile helps to manage complex projects. DevOps’ central concept is to manage end-
to-end engineering processes.

Task The agile process focuses on constant DevOps focuses on constant testing and
changes. delivery.

Implementation The agile method can be implemented within a The primary goal of DevOps is to focus on col-
range of tactical frameworks like ‘sprint’, ‘safe’ laboration, so it doesn’t have any commonly
and ‘scrum’. accepted framework.

Team skillset Agile development emphasises training all team DevOps divides and spreads the skillset
members to have a wide variety of similar and between the development and operations
equal skills. teams.

Team size A small team is at the core of agile. Because Relatively larger team size, as it involves all the
the smaller the team, the faster they can move. stakeholders.

Duration Agile development is managed in units of DevOps strives for deadlines and benchmarks
‘sprints’. This time is much less than a month with major releases. The ideal goal is to deliver
for each sprint. code to production daily, or every few hours.

Feedback Feedback is given by the customer. Feedback comes from the internal team.

Target areas Software development An end-to-end business solution and fast

delivery.

Shift-left principles Leverages shift-left Leverages both shift-left and shift-right.

Emphasis Agile emphasises on the software development DevOps is all about taking software that is
methodology to create software. When the ready for release and deploying it in a reliable
software has been developed and released, the and secure manner.
agile team will not care what happens to it.

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  63

Focus

Cross-functional Any team member should be able to do what’s
required for the progress of the project. Also,
when each team member can perform every
job, it increases understanding and bonding
between them.
Communication Scrum is the most common method of im-
plementing agile software development. Daily
Scrum meetings are carried out.
Documentation The agile method gives priority to the working
system over complete documentation. It is ideal
when you’re flexible and responsive. However,
it can hurt when you’re trying to turn things
over to another team for deployment.
Automation Agile doesn’t emphasise on automation.
Though it helps.
Goal It addresses the gap between customer needs,
and the development and testing teams.
Focus It focuses on functional and non-functional
readiness.
Importance Developing software is inherent to agile.
Speed vs risk Teams using agile support rapid change, and a
robust application structure.
Quality Agile produces better application suites with
the desired requirements. It can easily adapt
according to the changes made on time, during
the project’s life.
Tools used JIRA, Bugzilla and Kanboard are some popular
agile tools.
In DevOps, development teams and opera-
tional teams are separate. So, communication
is quite complex.
DevOps communications involve specs
and design documents. It’s essential for
the operational team to fully understand
the software release and its hardware/net-
work implications for adequately running the
deployment process.
In DevOps, process documentation is fore-
most because the software is sent to the
operational team for deployment. Automation
minimises the impact of insufficient docu-
mentation. However, in the development of
complex software, it’s difficult to transfer all
the knowledge required.
Automation is the primary goal of DevOps. It
works on the principle of maximising efficiency
when deploying software.
It addresses the gap between development +
testing and operations.
It focuses more on operational and business
readiness.
Developing, testing and implementation -- all
are equally important.
In the DevOps method, the teams must make
sure that the changes that are made to the
architecture never develop a risk to the entire
project.
DevOps, along with automation and early bug
removal, contributes to creating better quality
software. Developers need to follow coding
and architectural best practices to maintain
quality standards.
Puppet, Chef, TeamCity, OpenStack and AWS
are popular DevOps tools.

Challenges The agile method needs teams to be more pro- The DevOps process needs the develop-
ductive, which is difficult to manage every time. ment, testing and production environments to
streamline work.

Advantage Agile offers a shorter development cycle and DevOps supports agile’s release cycle.
improved defect detection.

ƒƒ T
he target area of DevOps is
software development whereas the
target area of agile is to deliver end-
to-end business solutions quickly.
ƒƒ DevOps focuses more on operational
and business readiness, whereas
agile focuses on functional and non-
functional readiness.
By: Harsukhdeep Singh
The author has worked as a QA
engineer for the last four years and is
an open source enthusiast.

64  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


Gulp: A DevOps Based Tool
for Web Applications
This article highlights the features of Gulp, an open source cross-platform
streaming task runner that lets software developers automate many
development tasks. It covers the installation of Gulp, and touches upon the
code required for task and module management.
S
oftware development teams
need to work collaboratively
on numerous tasks that are time
consuming. DevOps is very
popular in the corporate world for the
automation, collaboration and real-time
control that it enables on different types
of software projects. Many corporate
giants including Amazon, Oracle, Red
Hat, SaltLake, Nagios, etc, use DevOps
tools for real-world projects.
DevOps refers to the approach used
to correlate the software development
process with the various IT tasks to
provide better software quality and
streamlined deliveries. It speeds up the
development, testing and deployment of
code of high quality.
As the name indicates, DevOps
is a combination of the development
(Dev) and information technology
operations (Ops) of an organisation
that delivers software reliably, in a
shorter time. The main purpose of
DevOps is to automate the software
development life cycle. The key
features associated with DevOps are
speed, reliability and better quality.
Agile development and enterprise
systems management (ESM) make
up the foundation of DevOps. Agile
is linked to the software development
process and ESM covers various
processes that involve configuration
management, system monitoring,
etc. The need for DevOps arises
because the operational team and
development team work separately.
The design, development and testing
teams take more time and more errors
occur during deployment.
The architecture of DevOps
provides a combined approach
for development and operations.
This approach involves developing
code, testing, planning, monitoring,
deployment, operations and the
release. This architecture is followed
by large applications and those
that are hosted on cloud platforms.
This is because, in the case of large
applications where the development
and operational teams do not work
in synchronised environments,
long delays can occur in the
process of design, deployment and
testing. DevOps overcomes delays,
maintaining high quality and timely
delivery of the product.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  65
Focus
The DevOps life cycle consists of
seven phases which are: continuous
development, continuous integration,
continuous testing, continuous
monitoring, continuous feedback,
continuous deployment and continuous
operations.
ƒƒ Continuous development: This
includes a plan for the software and
development processes.
ƒƒ Continuous integration: This
consists of commit changes
and integrated code from teams
that have worked on the same
project and have performed
integration testing.
ƒƒ Continuous testing: During
this process, the team identifies
bugs in the code with the help of
automated testing, which is more
reliable and less time consuming
than manual testing.
ƒƒ Continuous monitoring: This
monitors all the processes that
take place in different phases of
the software development life
cycle and records information
regarding problems.
ƒƒ Continuous feedback: This
improves subsequent versions of the
software by removing what is not
relevant according to the customer.
ƒƒ Continuous deployment: This
involves code deployment on all
the servers.
ƒƒ Continuous operation: This is
the automation of the process and
key operations.
Focus

DevOps based free and Table 1: Prominent DevOps tools

open source tools Tool URL
DevOps provides a large variety
of tools that are useful in the Gulp https://gulpjs.com
automation of the software Worksoft https://www.worksoft.com
development, testing and release
processes. Table 1 lists some of Kamatera https://www.kamatera.com
these tools.
Buddy https://buddy.works
Gulp: An open source tool Nagios https://www.nagios.org
for Web applications
Gulp is used by software Chef https://chef.io
development teams as the streaming Jenkins https://www.jenkins.io
build system with which a number
of tasks can be programmed and Vagrant https://www.vagrantup.com
automated in an effective way. Splunk https://www.splunk.com
Gulp is an effective task runner
that has the strong base of the Git https://git-scm.com/
Node.js platform. It makes use of Puppet https://puppet.com
powerful JavaScript code for the
execution of tasks and corporate
level Web applications. JavaScript code Installing and working with Gulp
In Web development projects, ƒƒ C ode generation for cross- To install and work with Gulp, you
the following are a few of the browser compatibility need to install Node.js. Once Node.js is
tasks required to be worked on ƒƒ Deployment of files for staging installed, Gulp can be integrated with the
continuously by the software and uploading to servers following instruction:
development teams: Often, these tasks are repetitive
ƒƒ Formation, generation and in nature and require a lot of time. $ npm install --global gulp
transformation of templates Gulp is a powerful and multi-
ƒƒ Compression of multimedia files featured tool that can be used to Once the installation is finished, the
ƒƒ Code linting automate and speed up the tasks installed version can be verified as follows:
ƒƒ Code validation associated with software project
ƒƒ Integration of stylesheets and development. $ gulp -v

Figure 1: Official portal of Gulp Continued to page 69...

66  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Focus

DevOps is the Future

of Software Development

DevOps scores
over legacy,
monolithic and
agile software
development.
This article
discusses the
various stages
of the DevOps
software
development
cycle and
delineates the
appropriate
FOSS tools that
can be used
at each of the
stages.

D
evOps is a method of
software engineering that
combines both software
development (Dev) and
information technology operations
(Ops) to create high quality software
within a short period of time. DevOps
shortens the time taken for software
development through continuous
delivery and the integration of code.
As the name suggests, DevOps is
a combination of development and
operations. Traditionally, software
development is done by a team of
stakeholders comprising business
analysts, software engineers,
programmers and software testers.
The software development is done
by the development team through a
software life cycle comprising various
stages like customer requirement(s),
planning, modelling, construction
and deployment. However, such a
development cycle could take a lot
of time and collaborative effort to
successfully deliver software to the
customer. This is also known as agile
software development.
DevOps can help overcome the
drawbacks of the agile software
development life cycle. In the agile
method, to develop software, every
stage of the life cycle needs to be
completed prior to the final release
and that could take a lot of time for
the software to reach maturity. In the
case of DevOps, instead of delivering
the whole software, small chunks of
code are updated, and the updated
software is released to the operations
team continuously, which speeds up
the software development cycle. The
DevOps life cycle can be automated
using various development tools and
requires less manual activity. The
DevOps concept first originated in 2008
during a discussion between Patrick
Debois and Andrew Clay Shafer.
However, the idea only started to spread
in 2009 with the advent of the first
DevOps Days event held in Belgium.
Now, most tech giants like Facebook,
Google, Amazon, Netflix, etc, have
adopted the DevOps culture.

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  67

Focus
Build Code
Test Plan
Release
Deve
lopment
Continuous Delivery
Figure 1: DevOps life cycle
DevOps life cycle and tools
The DevOps life cycle consists of eight
stages – plan, code, build, test, release,
deploy, operate and monitor. There are
various free and open source software
(FOSS) tools that can be used to improve
and automate the DevOps life cycle.
The eight stages of DevOps are
explained below.
1. Plan: This is the starting stage of
DevOps where the planning for the
software development is done by
the development team. Everything
from the software requirements to the
development timeline is planned and
the team works accordingly. FOSS 5. Release: After the code is tested
tools like Redmine, Trac and Git are
used at this stage.
2. Code: After everything is planned,
the coding for the software
development is done. Coding can be
done from scratch or can be reused,
depending on the requirements. Many
FOSS tools can be used for coding.
Git is one such tool that is used to
automate the process.
3. Build: After the coding is done, it is
shared with other software engineers
of the development team. The code is 6. Deploy: This stage belongs to the
either approved or rejected after it is
reviewed. If it’s approved, then it is
merged with the main codebase of the
repository. FOSS tools like Gradle
can be used to automate this process.
4. Test: Once the new code is merged
68  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
Continuous Integration
Deploy
Dev Ops
Monitor
Operations
with the codebase, it is tested in a
virtual environment, using a VM
or Kubernetes. A series of both
manual and automated tests are
conducted at this stage. This is
the most crucial stage of DevOps
and should go through without
failure. This stage could also
cause bottlenecks and increase the
timeline. In order to complete the
stage successfully without much
delay, various continuous testing
tools are used. FOSS tools like
CruiseControl and Selenium are used
to automate the process.
successfully, it is prepared for
release. At this stage the development
team decides which features of the
software product should be enabled
or disabled by default and when it
should be released. This is the final
stage for the DevOps development
cycle, and after this the software/
code is delivered to the operations
team. This stage can also be
automated using FOSS tools like
Jenkins and Bamboo.
operations team and it starts after
the release of the software by the
development team. The software
is then deployed by the operations
team using various tools. The release
is configured according to the
requirements of the operations team.
FOSS tools like Puppet, Ansible and
Saltstack are used to automate the
deployment process.
Operate
7. Operate: After the software has
been updated and configured, the
operations team starts operating the
products and services with the updated
software. This is done by either using
proprietary or FOSS tools.
8. Monitor: This is the final stage for
the IT operations team and also for the
DevOps life cycle. Here the customer
requirements are gathered and the
data is sent to the development team
to update the software product/service
for the next iteration of DevOps.
FOSS tools like Nagios are used by
the operations team to automate the
monitoring process.
Benefits of DevOps
DevOps has many more benefits than
the agile software development process.
In case of the latter, the development
process progresses through four main
stages – planning, coding, testing and
release. The planning for the software
development is done based on customer
requirements and involves assigning
various tasks to the stakeholders,
like designing flowcharts, writing
documentation, estimating timelines, etc.
After the planning is done, the coding
for the software development starts
and a working prototype is created.
The prototype is then tested in the third
(testing) stage. The testing stage is the
most important part of the agile software
development life cycle because it can
affect the overall quality of the software.
This stage can also cause bottlenecks
because the time required for software
testing cannot be estimated precisely and
could go beyond the project’s timeline.
After successfully testing the
software, it is then released to the
customer in the fourth stage. However,
if the customer finds any problems with
the software, then the development
team takes the customer’s feedback for
another iteration of the development

cycle. Then the whole development
process takes place again, with the
four stages, to improve the software
and then it is released again, with
another version number. Once again
the customer’s feedback is taken, and
if any improvements are required then
the development cycle goes into the
iteration stage again and the cycle
keeps continuing until the software
reaches maturity (a stable software).
This type of development cycle could
take an indefinite amount of time and
causes delays in software development.
That’s why using DevOps can make
the development cycle quicker and
more effective.
Continue from page 66...
Module management in Web
applications using Gulp
The following is the hierarchy used for
the files and folders while working with
Gulp streaming applications.
src: Path of the pre-processed
HTML code files:
Scripts: Assorted pre-processed scripts
Styles: Multiple Stylesheet Source Code
in CSS format
Build: This folder is automatically
created with the dynamic generation of
production files:
Images: Compressed image files
Scripts: Script file with minified codes
Styles: CSS script with minified codes
gulpfile.js: Configuration file of
Gulp in JavaScript for defining the tasks.
Task management and
automation in Gulp
Gulp integrates the modular approach
with task management so that the
software development and related
phases can be programmed with
accuracy. You need to create a Gulp task
with the following structure:
With DevOps, the development
team communicates with the operations
team and does not deal with the
customers. In DevOps, only small
chunks of code are continuously
coded and tested, which prevents any
bottlenecks or delays in delivering the
code. DevOps provides continuous
delivery and integration of the code.
This is why it is better than agile
software development.
As DevOps has many benefits over
agile software development, many
software companies are resorting to the
DevOps culture. Because it provides
continuous delivery and integration,
DevOps can help software products
gulp.task(‘name of the task’, function()
{
// Operations
});
…where the name of the task
refers to the title of the task. All the
operations or tasks are defined in
function(). The ‘gulp.task’ is used to
register the function for the execution
of the operations.
Here is an example:
gulp.task(‘mytask’, function() {
var SourcePath = ‘src/myimages/**/*’,
DestinationPath = ‘build/myimages’;
gulp.src(SourcePath)
.pipe(changed(DestinationPath))
.pipe(mytask())
.pipe(gulp.dest(DestinationPath));
});
The execution or running of the
Gulp file is done using the following
instruction:
By: Dr Gaurav Kumar
The author is the MD of Magma Research and Consultancy Services, Ambala and is associated
with various universities, institutes and autonomous organisations for the delivery of expert
lectures, technical workshops and consultancy. Website: www.gauravkumarindia.com.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  69
Focus
and services to reach maturity
in a very short period of time.
DevOps is mainly suitable for the
development of cloud computing
products and services, because
it needs collaboration between
the development team and the IT
operations team. As there is a growing
demand for cloud computing products
and services, DevOps is the right
choice for most software companies
and has a great future.
By: Debojit Acharjee
The author is a software
engineer and writer.
$ gulp mytask
Today, large scale software
projects have diversified teams of
programmers, testing engineers as
well as other team members. Hence,
there is a need to automate and control
tasks effectively so that companies can
get fast delivery of the product, and
to provide an integrated development
environment with testing using the
different tools of DevOps. The main
requirement for the DevOps approach
towards software development is
that it should be cloud-centric so
that simultaneous operations can be
performed with the help of DevOps
based tools and technologies. It is
very difficult for small companies to
follow DevOps because they can’t
afford cloud costs. From the point of
view of a career, DevOps is in huge
demand in the software development
job market, with a very good salary
package on offer.
 CODE
SPORT
In this month’s column, we will first briefly discuss the
common patterns in coding interview questions, and then
feature a few of these questions.
T
here are typically a number of
patterns in coding interview
questions. Some of these include
dynamic programming, sliding
window, monotonic queue/stack, divide and
conquer patterns, etc. Problems that appear
difficult at first, turn out to be quite simple
once the underlying pattern can be identified.
Let us talk about a few examples of each of
these patterns.
Dynamic programming (DP) continues
to be quite popular in coding interviews.
Dynamic programming questions are
difficult unless you have practised earlier
on similar types of questions. While the
solution once framed into a DP equation looks
quite simple, there is quite a leap from the
problem statement to framing the dynamic
programming solution equation. One way
to navigate the gap, typically, is by thinking
about the problem first in terms of a top-
down recursive solution. Once we can find
the recursive equation, extend the solution by
applying memorisation so that you don’t have
to recompute the same sub-problems over
and over again. Once we have the memorised
solution, try and see if you can frame it as a
DP equation by looking at the sub-problems.
Let us look at a sample question.
There is a frog that needs to cross a
river. There are stones on the crossing path,
which are divided into units of one, on which
the frog can jump and cross the river. The
70  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
Sandya Mannarswamy
position of these stones is given as integers
in a sorted array. The frog is sitting on the
first stone, which is at position 0, and the first
jump must be of one unit. At any point during
the journey, when the frog jumps, it needs to
only land on a stone, else it can’t cross the
river. At any stone, it can only jump (k-1), k
or (k+1) units from the current stone, where
k is the size of the previous jump. The frog
needs to land on the last stone for it to cross
the river. You need to write code to determine
whether the frog can actually cross the river,
given an array representing the position of
the stones on the path.
The brute force solution would be to frame
this problem as a recursive solution. Given
the position of the current stone, and the size
of the last jump, we can recursively keep
checking whether we can reach a stone from
the current position, and from then onwards to
the next stone, and so on, until we end up at
the last stone or we run out of stones. Given
the size of the last jump, k, the possible sizes
of the new jump are (k-1), k and (k+1). We
need to check if there exists a stone at the
current position + the new jump size. If so,
we can then again recursively do the check
from that stone (with that stone as the starting
point and the jump size with which that stone
was reached). If we can reach the last stone
through any of the paths, the crossing was
successful. If there is no successful path to the
last stone, the frog fails to cross.

The brute force recursive solution has many
redundant computations since we can end up with the
same starting stone and jump size from many of our
recursive paths. Hence, an obvious improvement is to
memorise these computations. We can cache the result
of the recursive function for the stone index and jump-
size combination and if this same recursive call is made,
we can return the cached result instead of the recursive
function call.
As we compute the recursive solution, it becomes
clear that at each stone, we are trying to find out what
the possible jump sizes are from this stone. Once we
have the set of jump sizes known for each stone, we can
then check if there exists a stone at any of these jump
sizes. If there exists a stone with that position, we can
then update the new jump sizes for the newly reached
stone. If we end up reaching the last stone during our
traversal of all stones, then we have figured out that
the frog can cross successfully. Else the frog cannot
cross. We create a map which can remember the set
of possible jump sizes associated from each stone’s
position. Hence, given a stone and its set of jump sizes,
the DP solution computes for all reachable stones from
the current stone, and their possible jump sizes. Given
that this is a reachability problem, it is also possible
to formulate this in terms of graph reachability. Given
the starting stone and destination stone, the idea is to
find out whether a possible path exists between them
while obeying the constraints on the jump sizes. I will
leave it to our readers to come up with the depth first
search/breadth first search method of computing this
reachability solution.
Another pattern that appears often in coding interviews
is the sliding window pattern. Let us look at an example.
Given a source string S and a target string t, write a
function that can return the minimum length substring of
S which contains all the characters in the target. The brute
force approach is to enumerate all possible substrings
of S, see which of them contain all characters in target t
and choose the minimum length substring among these
substrings. Instead, we can apply a sliding window
approach to this problem.
Typically, a sliding window has a left pointer and
a right pointer. We first expand the right pointer on the
source string till the window includes all characters in
target t. Now we have a valid substring containing all
of the target characters, which is represented by the
window. We can now move the left pointer forward to
shrink the window so that we can reduce the length of
the string. We keep repeating this operation over the
complete source string and keep updating the minimum
length substring among all the valid substrings we have
encountered. I will leave it to our readers to write and
Guest Column CodeSport
test the actual code for this problem. The sliding window
pattern is typically applicable to problems that are over
the sequential single dimensional data structure such as
an array or a string, though they can be applied with some
modifications to matrix problems also. Here are a few
sample problems for our readers to practice.
(1) Given an array of integers A of size N, find the
contiguous sub-array with the minimum sum. A brute
force approach would be to consider all sub-arrays (which
is N^2), compute the sum for each of them and choose the
minimum. Can you do better than that?
(2) Given a binary array containing 1s and 0s, find the
length of the longest consecutive set of 1s in the array
if you can replace k 0s with 1s. First assume that k is 1.
Then solve this problem for any arbitrary ‘k’.
(3) Given K sorted arrays, write a function to form a single
sorted array from all these arrays.
(4) You are given an array of N integers. Write a function
that computes the maximum absolute difference between
the nearest smaller left element and the nearest smaller
right element among all the array elements. If there is
no left smaller element for a given array element, take
the left smaller element as 0 (and do the same for the
right smaller element).
(5) You are given an array of N integers, for which each
array element represents the height of a histogram.
Each of the histograms has unit widths. You are
asked to find the area of the largest rectangle in the
histogram. Remember that for any two histogram bars,
the smallest bar lying between them decides the height
of the rectangle.
(6) You are given an integer N. You need to find another
integer M that has the same set of digits present in N but
is greater in value than N. M should be the smallest such
integer. If no such integer exists, return -1.
Feel free to reach out to me over LinkedIn/email if you
need any help for your coding interview preparations. If
you have any favourite programming questions/software
topics that you would like to discuss on this forum, please
send them to me, along with your solutions and feedback, at
sandyasm_AT_yahoo_DOT_com.
I request our readers to follow the public health guidelines
and stay safe. Please don’t worry if you are not able to focus
fully on learning new technologies, programming, etc. These
are difficult times. So first take care of your mental and
physical health, help others wherever you can, and stay safe.
Wishing all our readers happy coding until next month!
Stay healthy and stay safe.
By: Sandya Mannarswamy
The author is an expert in natural language processing and is
currently working as an independent researcher. Her interests
include natural language processing, machine learning and AI.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  71
Developers Overview

A Study of Various Open

Source Blockchain Platforms
Blockchain platforms are important because they can be used in e-governance,
eliminate middle men, and secure the confidence of businesses. Get acquainted with
the various blockchain platforms in this interesting article.

T
he term ‘blockchain
technology’ typically refers
to the transparent, trustless,
publicly accessible ledger
that securely transfers the ownership
of units of value using public key
encryption and proof of work methods.
The technology uses decentralised
consensus to maintain the network,
which means it is not centrally controlled
by a bank, corporation or government.
In fact, the larger the network grows
and becomes increasingly decentralised,
the more secure it becomes. Blockchain
technology is picking up at a fast
pace nowadays. The technology that
emerged in 2009 as the underlying
platform for Bitcoin exchange has now
evolved into a mainstream technology.
It finds application in various fields
like healthcare and finance. Many
companies venturing into this domain
are developing blockchain based
applications that help in making
business operations more transparent
and efficient.
What exactly is blockchain?
At the most basic level, blockchain is
just a chain of blocks, but not in the
traditional sense of those words. When
we use the words ‘block’ and ‘chain’
in this context, we are actually talking
about digital information (block) stored
in a public database (chain).
Blocks on the blockchain are made
up of digital pieces of information and
they have three main parts.
Part 1: Blocks store information
about the transaction like the date, time
and amount involved in the transaction
of the recent purchase.
Part 2: These blocks store
information about the participants in
the transactions. For example, a block
for purchase of some items from
Amazon would record the name along
with other information to amazon.
com. Instead of using the actual name,
the purchase is recorded without
any identifying information using a
unique ‘digital signature’ like a sort of
user name.
Part 3: Blocks store information
that distinguishes them from other
blocks, much like names of people, and
each block stores a unique code called
a hash that allows every block to be
different from the other. Hashes are the
cryptographic codes created by special
algorithms.
Blockchain variants
Public: The ledgers of these
blockchains are visible to everyone
on the Internet. They allow anyone to
verify and add a block of transactions
to the blockchain. Public networks have
incentives for people to join and are

72  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


free for use. Anyone can use a public
blockchain network.
Private: Private blockchains exist
within a single organisation, allowing
only specific people of the organisation
to verify and add transaction blocks.
However, everyone on the Internet is
generally allowed to view.
Consortium: In this blockchain
variant, only a group of organisations
can verify and add transactions. Here,
the ledger can be open or restricted to
select groups. A consortium blockchain
is used across organisations. It is only
controlled by pre-authorised nodes.
Pillars of blockchain platforms
When there are many public and private
blockchain platforms available in
the marketplace, one has to carefully
choose a suitable option based on one’s
requirements. This has to be evaluated
based on the pillars of the blockchain
platform as listed below:
ƒƒ Decentralised network
ƒƒ Platform security
ƒƒ Immutability of the record state
Decentralised network: One of
the key architectural principals of the
blockchain platform is its decentralised
nature. This means the transaction
in the blockchain network is copied
across all the nodes of the network
Governance and Free structure
Principle and Business Goals
Transparency in Transaction
Decentralized Network
Platform Security
Platform Scalability
Figure 1: Pillars of blockchain platforms
and all the nodes are connected. This
blockchain platform is highly reliable
as there is no possibility of tampering
with the transaction record because
it is copied to all the nodes — and
tampering with all the nodes in the
network is practically impossible at
any given time.
Platform security: Though the
blockchain platform is decentralised in
nature with many users being a part of
the workflow process and participating
in the various stages of executing the
transaction, there is a higher level of
security ensured in any blockchain
platform due to its decentralised nature
and multi-node record copy. Also,
different blockchain platforms ensure
higher levels of platform security like
permissionless ledgers, the consensus
algorithm, usage of cryptocurrency
for transactions and the smart contract
facility, to name a few.
Record immutability: In any
blockchain platform, the decentralised
network and ledger copied across all
the nodes of the network ensures that
all the records kept in the ledger for
any transaction are secure enough so
that a change in a record is accepted
only if it’s accepted by all participants
across nodes, thus changing the record
unanimously across all the ledger
Record Immutability
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  73
Overview Developers
copies in the network. This ensures that
the records in the transaction processing
stages are immutable in nature. Any
correction in the record is amended as a
new transaction processing stage in the
network, thus ensuring all the record
processing transaction stages are kept as
distinct entries in the ledger.
Blockchain platforms
Blockchain platforms assist us in
creating applications that implement
the concepts of blockchain. Every
person or company does not have
either the resources or the time to
develop their own blockchain from
scratch, and hence these companies
leverage blockchain platforms
developed by tech giants for faster and
easier application development. Any
organisation deciding to implement
blockchain may choose from the
prominent frameworks available,
i.e., Ethereum, Hyperledger Fabric,
Quorum, Corda, Ripple, etc. The final
decision should be based upon the
suitability of each, to the organisation.
Let’s now discuss the most popular
open source blockchain platforms.
Ethereum
Ethereum (a public blockchain network)
was developed by Vitalik Buterin and
is considered an efficiently developed
platform that has smart contract features,
flexibility and multi-industry adaptability.
Ethereum acts as a base component in
building and developing most of the
decentralised applications. ERC20 is
the most popular token standard among
cryptocurrencies. Stability, security,
corruption prevention and zero downtime
are some of the benefits offered by
Ethereum over other applications.
The Ethereum network has two
types of accounts, namely:
ƒƒ External accounts
ƒƒ Contract accounts
Both are referred to as ‘state
objects’ and comprise the ‘state’ of the
Ethereum network, i.e., every state
object has a well-defined state. For
Developers Overview

Figure 3 explains the architecture

blocks of the OpenChain platform,
Mining which contains multi-layer components
Node for consensus, ledger and platform
services, and supports membership
services to handle workflow and
Ethereum Virtual Machine
permissioned transaction management.
ƒƒ Anyone can spin up a new
Smart Contract
OpenChain instance within seconds.
ƒƒ The administrator of an OpenChain
Block-1 Block-2 Block-3 ...... Block-n instance defines the rules of the
ledger.
ƒƒ End users can exchange value on
the ledger according to those rules.
ƒƒ Every transaction on the ledger is
Consensus
Ledger digitally signed, like with Bitcoin.

Figure 2: Ethereum architecture How OpenChain is different

ƒƒ Tokens on OpenChain can be
pegged to Bitcoin, making
Participants Network Transaction Contact it a side chain.
ƒƒ Has smart contract modules and
unified API.
Consensus Ledger ƒƒ Assigns aliases to users instead of
Membership Secure
Registry
using base-58 addresses.
Services ƒƒ Multiple levels of control and
Store Protocol an hierarchical account system
allowing permissions to be
at any level.
Platform Infrastructure ƒƒ Ability to have multiple OpenChain
instances replicating from each
Figure 3: OpenChain’s platform architecture other.

external accounts, the state comprises Key highlights Key highlights

the account balance while for contract
accounts, the state is defined by the
memory storage and balance.
How Ethereum is different
ƒƒ Ethereum offers greater
transparency as compared to
Hyperledger.
ƒƒ The flexibility of the platform
allows any developer to create
applications using an inbuilt
programming language.
ƒƒ Having ‘Ether’ as a built-in
cryptocurrency provides a
competitive edge to Ethereum
applications over Hyperledger and
Corda in use cases that require
cryptocurrency.
ƒƒ Activity: Highly Active on
GitHub
ƒƒ Type of network: Public, smart
contract based
ƒƒ Pricing: For transactions and for
computational services
ƒƒ Supported languages: Python,
GO, C++
ƒƒ GitHub repo: pyethereum
(Python); gpethereum (GoLang);
cpp-ethereum (C++)
OpenChain
OpenChain is an open source
distributed ledger technology. It is
suited for organisations wishing to
issue and manage digital assets in a
robust, secure and scalable manner.
ƒƒ Activity: Medium active on GitHub
ƒƒ Type of ledger: Distributed ledger
technology and is private
ƒƒ Pricing: Open source
ƒƒ Supported languages: JavaScript
ƒƒ GitHub repo: Openchain-js
BigChainDB
BigChainDB is built from a federation
of enterprise-ready database nodes,
such as MongoDB instances, which
store immutable information about
assets in a synchronised manner.
BigChainDB is a MongoDB
database that uses Tendermint to obtain
its blockchain features. A BigChainDB
network may be public, private or
permissioned, according to the access

74  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


permissions entities have over the
system. In a public BigChainDB,
any participant is able to access the
network or deploy that participant’s
own MongoDB+Tendermint node and
connect it to the database federation,
while a permissioned BigchainDB
could be managed by a consortium or a
governing entity, where every member
of the consortium manages their own
node in the network and no one can join
without permission.
BigChainDB’s transaction model is
analogous to that of Bitcoin as shown
GUI
Interfacing Layer
Node
Consortium
Node
Node
Member
Node
Blockchain Platform
Figure 4: BigChainDB architecture
Node
Node
Figure 5: Hydrachain architecture
in Figure 3, in the sense that an asset
transaction receives an asset input,
which is then transformed to an
output that may be used in the future
as an input for a new transaction.
Asset outputs can only be used once
as input for a transaction. There
are two types of transactions in
BigChainDB, as given below.
Create transactions generate
a new asset in the system (as a
JSON document in MongoDB)
with two types of information in
it. These are asset information,
CLI
Driver
Cluster
Validations
Store
Ledger
Node Node
Node Node
User
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  75
Overview Developers
which is immutable and can’t be
modified once the asset is created;
and metadata, which can be modified
through subsequent transfer
transactions.
Transfer transactions allow the
transfer of ownership of an asset or
the modification of the metadata.
The only one entitled to perform this
transaction over an asset is its owner.
These transactions use as input an
unused output of the asset, generating
as a result a new output with the
corresponding modifications.
Key highlights
ƒƒ Activity: Active on GitHub
ƒƒ Type of ledger: Multi-ledger
integration
ƒƒ Pricing: Open source
ƒƒ Supported languages: JavaScript
and Python
ƒƒ GitHub repo: bigchaindb
HydraChain
HydraChain, an extension of
the Ethereum platform, is fully
compatible with the Ethereum
Protocol on an API and contract
level. It supports distributed ledgers
and private chains, which are mainly
set up for the financial industry.
The infrastructure of HydraChain
allows users to develop smart
contracts in Python, which can
improve development and debug
efficiency enormously.
HydraChain has a well-defined
configuration system, as shown in
Figure 5, which provides flexible
customisation adjustments such as
transaction fees, gas limits, genesis
allocation or block time.
Features
ƒƒ Fully compatible with the
Ethereum Protocol
ƒƒ Accountable validators and native
contracts
ƒƒ Fully customisable, easy to
deploy, and open source with
commercial support available
Developers Overview

Key highlights
ƒƒ Activity: Low but actively updated Services Contracts Whitelist Blacklist
on GitHub
ƒƒ Type of ledger: Private Flow Services
ƒƒ Pricing: Open source
ƒƒ Supported language: Python Blockchain Platform
ƒƒ GitHub Repo: Hydrachain (Python)
Assets Transaction Store
Corda
Corda is an open source blockchain
platform that enables businesses to Members Workflow Rules
transact directly and in strict privacy
using smart contracts, reducing Key
transactions and record-keeping Vault Identity Nodes
Management
costs while streamlining business
operations. In a world of permissionless Approval
Messaging Client Persistent DB
Process
blockchain platforms, in which all
data is shared with all parties, Corda’s
strict privacy model allows businesses Figure 6: Corda architecture
to transact securely and seamlessly, as
shown in Figure 6.
User Interface
R3 delivers two interoperable and
fully compatible distributions of the
platform – Corda, a free download
based on the code available on GitHub; Services Layer Data Store
and Corda Enterprise, an enterprise
blockchain platform that offers features
and services fine-tuned for modern-day
businesses.
Corda provides three main tools to Node 1 Node 2 Node 3 Node 4 Node 5 Node 6
achieve global distributed consensus:
ƒƒ Smart contract logic, which
specifies constraints that ensure
state transitions are valid according
to pre-agreed rules, described in the Assets Transaction API Permission
contract code as part of CorDapps.
ƒƒ Uniqueness and timestamping Blockchain Platform
services known as notary pools to
order transactions temporally and
eliminate conflicts. Figure 7: MultiChain architecture
ƒƒ A unique component called the flow ƒƒ S upported language: Kotlin MultiChain is for creating new
framework, which simplifies the ƒƒ GitHub repo: Corda blockchains with their own native
process of writing complex multi- currencies and/or issued assets.
step protocols between multiple MultiChain Users cannot transact existing
mutually distrusting parties across MultiChain technology is a platform cryptocurrencies on MultiChain unless
the Internet. that helps users to establish a certain someone trusted acts as a bridge in the
private blockchain that can be used by middle, holding some cryptocurrency
Key highlights organisations for financial transactions. and issuing tokens on MultiChain to
ƒƒ Activity: Actively updated on A simple API and a command-line represent it as shown in Figure 7.
GitHub interface are what MultiChain provides MultiChain is an off-the-
ƒƒ Type of ledger: Distributed us. This helps to preserve and set shelf platform for the creation
ƒƒ Pricing: Open source up the chain. and deployment of private

76  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Overview Developers

Features/Platform Ethereum OpenChain BigChainDB HydraChain Corda Multichain

Security Spying data is not
possible
Support/Governance Ethereum Consortium Hierarchical system to set Federation Consensus
Community Activity Democratic
Autonomous
Organizations (DAO)
Limitations Open financial system Unified API support and
access
Industry Focus Multi-Industry
application
Type of Ledger PermissionLess
Functionality Digital Asset
Management
Smart Contract Ethash - Proof of
Work (PoW)
Crypto Currency used ETH
Figure 8: Comparison of open source blockchain platforms
blockchains, either within or between
organisations. It aims to overcome
a key obstacle to the deployment
of blockchain technology in the
institutional financial sector, by
providing the privacy and control
required in an easy-to-use package.
Key highlights
ƒƒ Activity: Medium but actively
updated on GitHub
ƒƒ Type of ledger: Private,
permissioned
ƒƒ Pricing: Open source
ƒƒ Supported languages: Python, C#,
JavaScript, PHP, Ruby
ƒƒ GitHub repo: savior (Python); c#
MultichainLib (C#); Multichain-
Node (JavaScript);
libphp-multichain (PHP);
multichain-client (Ruby)
Functionality
The functional architecture of the
blockchain platform is based on
various consensus algorithms, which
are used as the basis of the platform
execution mechanism. These include
the following.
Proof of Work (PoW): This
is the original type of consensus
algorithm from Satoshi Nakamoto’s
first research on blockchain. This
algorithm is used as a mechanism to
confirm the transaction completeness
and if successful, a new block gets
added to the chain of network. This
Multiple levels of control Highly secure with Extension of Ethereum Higher privacy for Moderate security with
transaction level platform architecture data and transaction privacy features for
permissions transaction
Ethereum consortium Corda Consortium Global consortium
permission at any level Model
Slack community develop- Community develop- Open community Slack and StackO- Git based community
ment ment in Git verflow work
Data oriented solution In presence of pending No global broadcast Flexible to extend
extensibility is not available and no industry focus transaction only a new of transaction across privacy features is not
block gets created network supported
Insurance and Banking Multi-Industry ap- Financial services Banking, Financial Multi-Industry ap-
plication Services, Insurance plication
PermissionLess Permissioned or Permissioned Permissioned Permissioned or
private network Private network
Digital Asset Management Big Data management Customizable fee, genisis Pluggable consensus Data Ledger functions
allocation, block time
Partionned Consensus Federation of Voting Proof of Work (PoW) Supported Proof of Work (PoW)
nodes
No Native currency. No Native currency. ETH No Native currency. No Native currency.
helps any Distributed Denial of When any transaction happens
Service (DDoS) network attacks. in a blockchain platform, it gets
Proof of Stake (PoS): Since PoW validated either by permissionless
involves a lot of system resources like or permissioned blockchain. In
GPU, it is an advanced mechanism permissionless blockchains, the
suitable for sophisticated blockchain transaction is validated by the public,
networks. PoS solves the challenge of who are participants in the network
this high resource usage by using an and hence the transaction processing
algorithm based on distributed tokens rate is higher at any given time.
and static coin supply. In permissioned blockchain, the
Deleted Proof of Stake (DPoS): transaction is validated by a selected
This is an extended version of PoS. group of participants and approved by
The PoS is like a lottery mechanism the blockchain owner or approver.
based on the coin stake of the users The table in Figure 8 compares
and DPoS is based on influencing the various features and architectural
network by all stakeholders equally. It points of the blockchain platforms like
is based on an algorithm approach that security, support, community activity,
balances electing a list of nodes by any limitations in platform support, industry-
stakeholder to complete the transaction. focus of the solution and the type of
There are other consensus ledger used in the platform, as well as
mechanisms like Practical Byzantine the functionality supported.
Fault Tolerance (PBFT) and Directed
Acyclic Graph (DAC), which are
evolving in the Blockchain 3.0 wave References
(Blockchain 1.0 is Bitcoin architecture [1] https://ethereum.org/
[2] https://www.openchainproject.org/
and Blockchain 2.0 is Ethereum [3] https://www.bigchaindb.com/
consensus architecture). They are [4] https://www.hyperchain.cn/en
getting implemented in platforms like [5] https://www.corda.net/
[6] https://www.multichain.com/
HyperLedger and Ripple.
By: Anand Nayyar and Dr Magesh Kasthuri
Anand Nayyar works at Duy Tan University in Vietnam. He loves to explore open source
technologies, sensor communications, network security and the Internet of Things.
Dr Magesh Kasthuri is a Ph.D in artificial intelligence and the genetic algorithm. He
currently works as a senior distinguished member of the technical staff at Wipro
Technologies.

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  77

Developers Insight

A Few Surprising Programming

Language Features
The first programming language that you learn leaves an indelible mark on you. If you are
used to coding in a particular language and then decide to take up a new one, the latter may
throw up a few surprises. Some of these can be fun but they could also be frustrating.

I
learned C more than 20 years ago
and as a C programmer at heart, I
found it difficult to adjust to other
languages. Nevertheless, I had
to learn or use many others like C++,
Java, x86 assembly language, Python,
etc, over the years. I am no expert in
all these programming languages. If I
can order food, ask for water and call
a taxi in some (natural) language, I
assume proficiency in that language.
Often, the same standards apply while
claiming proficiency in a programming
language. Recently, while learning
Haskell, a functional programming
language, I came across a feature called
lazy evaluation of expressions (to be
discussed later in this article), and I was
literally surprised. This is something
that has happened to me a lot while
learning new programming languages.
While learning Python, it was a surprise
to learn that you don’t need to explicitly
declare the type of a variable before
storing data into it.
This article discusses some
of the unique features of different
programming languages that might
surprise programmers well versed in
some other programming language.
Now let us also try to find out why this
occurs. Major surprises tend to occur
when a programmer learns a language
that has a different paradigm. For
example, a Java (an object-oriented
programming language) programmer
learning Haskell (a functional
programming language) might come
across a large number of features that
are surprising and a few which might
even look counter intuitive. Similarly,
compiled and interpreted languages
often differ a lot in their features. An
example for this is a C++ (a compiled
language) programmer learning Python
(an interpreted language).
Similar surprises might await
a person switching from a general-
purpose programming language
to a domain-specific one — for
example, a C (a general-purpose

78  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


programming language) programmer
learning JavaScript (a domain-
specific programming language).
The programming experience might
also depend on the underlying
architecture and operating system. An
x86 assembly language programmer
(from Intel) learning MIPS assembly
language (from MIPS Technologies)
or a PowerShell (from Microsoft) user
learning Linux shell scripting are bound
to encounter a few surprises. Of course,
there could be many other reasons for
this, but the above mentioned ones
seem to be the most obvious.
For those learning their first
programming language, it’s most
probable that none of the features are
surprising. Now let us look at a few
features of programming languages that
might surprise someone who learns these
as their second programming language.
Complicated pointers in C/C++
Most undergraduate programmes in
computer science do have a course on
C programming and, to many, the most
difficult section is the one on pointers.
To add to the misery, you can declare
pointers of arbitrary complexity. As
an example, consider the C program
named pointer.c given below. This and
all the other programs discussed in
this article can be downloaded from
opensourceforu.com/article_source_
code/June20surprisinglanguage.zip.
#include<Stdio.h>
int main()
{ #include<iostream>
int *(*(* ptr)(int *))[2];
int *******ptr1;
int *******************************
*******************ptr3;
return 0;
} for(int i=0;i<5;i++)
The above C program compiles
without any errors. The pointer
variable ptr1 is a pointer to a function
that accepts a pointer to an integer
as an argument and returns a pointer
to an array of pointers to integers.
The pointer variable ptr2 is a pointer
to a pointer to a pointer to a pointer
to a pointer to a pointer to a pointer
to an integer (I hope I have counted
correctly!). There are 50 asterisk
symbols before the declaration of the
pointer variable ptr3. But is there a
limit to this? I tried up to 1000 asterisk
symbols and it was still compiling fine.
I believe there are no upper limits set
by the C standard. There may be a limit
at which the C compiler might fail to
handle this, but I don’t know where
that limit is. No programmer will ever
use these sorts of pointers in a real
program. Nevertheless, these features
are available for us to use. A similar
C++ program will also compile without
any errors. Now back to our business.
Imagine the horror of a Java, Python or
Haskell (all of which are programming
languages that do not use pointers)
programmer who comes across such
monstrosities.
Confusing Octal numbers in C/
C++/Java
This may not be as big a surprise as
the previous one. But once, a long
time back, this feature of C/C++/Java
gave me quite a headache and I believe
we should discuss it. Consider the
C++ program octal.cc given below.
Similar C and Java programs (octal.c
and Octal.java) are also available for
download. What is the output of the
program octal.cc?
using namespace std;
int main()
{ loop with an else part. Consider the
int num[ ]={001,005,025,125,625};
{ for i in range(5):
cout<<num[i]<<endl;
} else:
return 0;
} for i in range(10):
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  79
Insight Developers
The array num contains the
first five numbers in the geometric
sequence starting at 1 with common
ratio 5. Figure 1 shows the output of
the program octal.cc. But why is the
number 21 printed instead of 25? Well,
in C, C++ and Java, a number like
0123 is treated as an octal number and
‘0x123’ is treated as a hexadecimal
number. In the program octal.cc the
numbers 001, 005 and 025 are treated
as octal numbers due to this reason. But
the numbers 001 and 005 are the same
in decimal and octal number systems,
whereas the number 025 in octal is 21
in decimal. Thus, the sequence printed
is 1, 5, 21, 125, 625. This feature is
convenient on many occasions, but
could lead to potential bugs if one is
not careful. For example, a Python
programmer who is familiar with the
notation ‘0o25’ might consider ‘025’ as
the number 25 with a leading zero.
Figure 1: Output of the C++ program octal.cc
The for loop with an else in
Python
As mentioned earlier, a C/C++/Java
programmer newly learning Python
will be surprised that explicit type
declaration is not required in Python.
Since this dynamic typing feature
of Python is well known to many
programmers, I will discuss a simple
yet surprising feature of Python; for
Python script loop.py shown below.
print(i)
print(“Noraml Exit from for loop”)
Developers Insight
print(i)
if i == 4:
break
else:
print(“Break from for loop”)
Figure 2 shows the output of the
Python script loop.py. Notice that the
else part of the for loop gets executed
only when exited normally from the
loop and not when exited through a
break statement. Though not essential,
for-else is a convenient feature that
is absent in most of the programming
languages.
Figure 2: Output of the Python script loop.py
Fork bombing with a Linux
shell script
What is the smallest program (in terms
of source code size) in any language
that comes to your mind, which when
executed will crash your system? I am
sure the winner will be the following
shell script.
x( ) { x | x & }; x
It defines a function called x, which
calls the function x itself recursively
and then pipes this result to another
recursive call to x in the background.
The fourth x in the script is a call to
the function x to begin the bombing.
Soon your CPU will be allocating all
its time to process just these calls to
the function x. Just 11 non-white space
characters in the script and your system
is down, so imagine the surprise of
those programmers whose favourite
80  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
programming language takes hundreds
of characters just to print the message
‘Hello World’ on the screen.
Warning! If you execute this code
on a Linux terminal and do not close
the terminal immediately, your system
will hang. In such a situation, you will
have to force restart your system.
Case-insensitive function
names in PHP
Imagine copying the directories named
‘SONGS’, ‘Songs’ and ‘songs’ from
your Linux machine to your friend’s
Windows machine. The Windows file
system in general is not case-sensitive
and you will be forced to rename 2 of
these directories before copying them,
because all the three directory names
(SONGS, Songs and songs) are the
same if treated in a case-insensitive
manner. Some programming languages
behave like this. A classic example is
PHP. Consider the PHP script named
case.php given below:
<!DOCTYPE html>
<html>
<body>
<?php
function PRINTMSG( ) {
echo “I AM PHP <br>”;
} of the script type.js?
printmsg( );
PrintMsg( );
PRINTMSG( );
pRiNtMsG( );
PrInTmSg( );
?>
</body>
</html>
Function names in PHP are
case-insensitive and the lines of code
printmsg(), PrintMsg(), PRINTMSG(),
pRiNtMsG(), and PrInTmSg() are all
calling the function PRINTMSG().
Figure 3 shows the output of the PHP
script case.php. Do notice that PHP
variable names are case-sensitive like
most other programming languages.
I AM PHP
I AM PHP
I AM PHP
I AM PHP
I AM PHP
Figure 3: Output of the PHP script case.php
Implicit type conversion in
JavaScript
Imagine the case of a Java programmer
learning JavaScript. Due to the
similarity in names we might think that
this will be an easy task. Even though
the syntax is somewhat similar, the
transition from Java to JavaScript is
not that easy. Java is a strongly typed
language where type checking is very
rigorous, while on the other hand,
JavaScript is a weakly typed language
with extensive implicit type conversion.
This is one feature of JavaScript that
should be avoided if possible. To
understand the pitfalls in the implicit
type conversion of JavaScript, let us
go through the JavaScript script named
type.js given below. What is the output
<!DOCTYPE html>
<html>
<body>
<script>
a = ‘2’ + 1
b = ‘2’ - 1
document.write(a + “<br>”);
document.write(b + “<br>”);
c = ‘1’ + 2 + 3
d = 1 + 2 + ‘3’
document.write(c + “<br>”);
document.write(d);
</script>
</body>
</html>

A seasoned Java programmer
will expect a number of errors in the
above code. But everything is fine with
JavaScript. Figure 4 shows the output of
the script type.js. Why does the variable
a have the value 21 and variable b
have the value 1? This is due to the
implicit type conversion in JavaScript.
The operator ‘-’ performs just one
function, a mathematical subtraction.
When a string and a number are the
operands of the ‘-’ operator, JavaScript
converts the string to a number. So in
the line of code, b = ‘2’ - 1, the string
‘2’ is converted to the number 2, and
the number 1 is subtracted from it to
obtain 1. Notice that, here, the variable
b contains a number, 1. Now let us look
at what happens with the operator ‘+’.
The operator ‘+’ performs two
functions, mathematical addition and
string concatenation. When a string
and a number are the operands of the
‘+’ operator, instead of converting the
string to a number, JavaScript converts
the number to a string. So in the line
of code a = ‘2’ + 1, the number 1 is
converted to the string ‘1’, and the
strings ‘2’ and ‘1’ are concatenated to
give ‘21’. Notice that here, the variable
‘a’ contains a string, ‘21’. With that
21
1 evaluation is often called ‘call-by-need’.
123
33
Figure 4: Output of the JavaScript script type.js
Figure 5: Output of GHCi interactive environment
knowledge, can you explain why the
variable ‘c’ contains the string ‘123’
and the variable ‘d’ contains the string
‘33’? I will give you a hint — the
associativity of the operator ‘+’ is what
matters. The operator ‘+’ has left-
associativity. Hence in the line of code
c = ‘1’ + 2 + 3, the operation ‘1’ + 2 is
carried out first, resulting in the string
‘12’ because ‘1’ is a string. Then the
expression becomes ‘12’ + 3, resulting
in the final string ‘123’ stored in the
variable c. But in the line of code d =
1 + 2 + ‘3’, due to left-associativity
of the operator ‘+’, the first operation
performed is 1+2, resulting in the
number 3, because both 1 and 2 are
numbers. Then the expression becomes
3 + ‘3’, resulting in the final string ‘33’
stored in the variable d. Imagine the
horror of a Java programmer going
through these results in his favourite
programming language!
Lazy evaluation in Haskell
Haskell is a general-purpose purely
functional programming language, which
has a feature called lazy evaluation!
Because of lazy evaluation, expressions
are not evaluated immediately if they
are just bound to variables. Instead, the
evaluation of an expression is carried
out only when its results are needed by
other operations. For this reason, lazy
Lazy evaluation enables infinite functions
to be stored in Haskell lists. Figure 5
shows the processing of one such infinite
list in the GHCi interactive environment.
The list ‘x’ represents the infinite list of
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  81
Insight Developers
natural numbers by the line of code let x
= [1..]. Notice that ‘x’ is not evaluated
at this point. The line of code take 10
x gives the list [1,2,3,4,5,6,7,8,9,10]
as output (the first ten elements of the
list x). But the third line of code print
x will lead to the complete evaluation
of the list ‘x’ resulting in printing the
natural numbers (from which you
have to forcefully exit). Notice that the
lazy evaluation feature of Haskell is
very powerful but at the same time has
received some criticism from experts.
Multiple main methods in Java
When I first learned Java
programming, I did a very poor job
because I tried to learn some advanced
features (Swing and AWT) without
mastering the basics. Till recently, I
was under the false impression that
a Java program can contain only
one main method. But then I got a
rude shock — it is possible to have
more than one main method in a Java
program. For example, consider the
Java program AAA.java shown below
with two different classes AAA and
BBB, each having one main method.
class AAA
{
void disp()
{
System.out.println(“Hello From
AAA...”);
}
public static void main(String[ ]
args)
Developers Insight

{ You can imagine the number of errors

AAA a = new AAA(); they are going to make within a short span
a.disp( ); of time. At least, that is what happened
} to me and it was very frustrating. Figure
} 7 shows an example for array indexing
in the Scilab console. I think the figure
class BBB clearly illustrates the matter. In addition
{ Figure 6: Output of the Java program AAA.java to MATLAB and Scilab, there are other
public static void disp() programming languages like Fortran,
{ --> x= [111 222 333]; Julia, Mathematica, etc, in which arrays
System.out.println(“Hello from start at index 1.
BBB...”); --> x= (0); A lot more could have been added
} to this discussion, but since I don’t
public static void main(String[ Invalid index. know whether or not these so called
] args) surprises are indeed surprising to
{ --> x= (1); others, let us stop for the time being.
BBB b = new BBB(); ans = The selection of surprising features is
b.disp( ); based on my personal experiences and
} preferences, so it might be quite possible
} 111. that the very universally surprising
feature Y of programming language X
Figure 6 shows the output of the Figure 7: Output on the Scilab console might be missing on this list. But the
program AAA.java. From the figure, it most important takeaway from this
is clear that the name of the class you computer scientist Edsger W. Dijkstra discussion is that while learning a new
use to invoke the Java Virtual Machine has convincingly argued that this should programming language, if you are really
(JVM) decides the main method to be the case and it would be nice if you surprised by a particular feature, then it
be called. You can see that the output could go through his paper). But after a might be the reason why you are learning
is different when you execute the while, the students master the language that language. All the features present
commands java AAA (the main method and become comfortable with array in your old programming language will
of AAA is called) and java BBB (the indices starting at 0. Years later, array make the learning of the new one easy,
main method of BBB is called). Though indices starting at 0 become second but the surprising new features will test
surprising, it reminds us that the main nature to them. And then one day they you and might even give you a career
method is no way special. have to learn and use MATLAB/Scilab boost. So, the next time you come across
for some mathematical computations. a surprising feature in a programming
Matrix indexing in MATLAB/ All of a sudden, they realise that language, be ready; mastering it might
Scilab indexing starts at 1 for MATLAB/Scilab. be the break you need.
Imagine students learning their first
programming language, be it Python,
Java, C, or C++. The first time they By: Deepu Benson
come across arrays, I am sure they will The author is a free software enthusiast whose area of interest is theoretical computer
have some difficulty adjusting with science. The open source tools of his choice include ns-2 and ns-3. He maintains a
technical blog at www. computingforbeginners.blogspot.in.
the indexing starting at 0 (the famous

Visit ‘Online Only’ expo-cum-conference

What’s New In Electronics & IoT from the comfort of your HOME…

During These Challenging Times?

VISIT NOW https://IndiaTechnologyWeek.com JUNE EDITION: 17th – 19th JUNE, 2020

What’s New In Electronics & IoT

82  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
 Let's Try Developers

Image Feature Processing

in Deep Learning using
Convolutional Neural Networks:
An Overview
David H. Hubel and Torsten Wiesel laid the foundations for building the CNN
(convolutional neural network) model after their studies and experiments on the human
visual cortex. Since then, CNN models have been built with near human accuracy. This
article explores image processing with reference to the handling of image features in
CNN. It covers the building blocks of the convolution layer, the kernel, feature maps and
how the activations are calculated in the convolution layer. It also provides insights into
various types of activation feature maps and how these can be used to debug the CNN
model to reduce computations and the size of the model.

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  83

Developers Let's Try
“M
ore than 50
per cent of
the cortex, the
surface of the
brain, is devoted to processing visual
information,” said William G. Allyn, a
professor of medical optics.
The human visual cortex is
approximately made up of around 140
million neurons. In addition to other
tasks, this is the key organ responsible
for detecting, segmenting and
integrating the visual data received by
the retina. It is then sent to other regions
in the brain where it is processed and
analysed to perform object recognition
and detection, and subsequently retain it
as applicable.
In the early 1950s, David H. Hubel
and Torsten Wiesel conducted a series
of experiments on cats and won two
Nobel prizes for their amazing findings
on the structure of the visual cortex.
They revolutionised our understanding
of the human visual cortex and paved
the way for much further research based
on this topic.
Their studies showed that in the
visual cortex, many neurons have
small local receptive fields. A local
receptive field means that these
neurons react to visual stimuli only if
they are located in a specific region
of the visual field of perception. In
other words, these neurons are fired
or activated only if the visual stimulus
is located at a particular place on the
retina or visual field. They found that
some neurons have larger receptive
fields and they are fired, activated or
react to complex features in the visual
field, which in a way are a combination
of low-level features to which other
neurons react. The low-level features
mean horizontal and vertical lines,
edges and corners and different angles
of lines, while high-level features are
the simple or complex combinations
of these low-level features. Figure
1 illustrates the local receptive
fields of different cells in the human
visual cortex.
84  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
Hubel and Wiesel also discovered
that there are three types of cells in
the visual cortex and each has distinct
characteristics, based on the features
they learn or react to. These are: simple,
complex and hyper-complex. Simple
cells are responsible for learning
basic features like lines and colours;
complex cells for learning features
like edges, corners and contours, while
hyper-complex cells are responsible
for learning combinations of features
learnt by simple and complex cells.
This powerful insight paved the way
to understanding how perception is
built – the receptive fields of multiple
neurons may overlap and together they
tile the whole field. These neurons
work together, with a few neurons
learning features on their own and
others learning by combining the
features learnt by other neurons, and
finally integrating to detect all forms of
complex features in any region of the
visual field.
For the video of the Hubel and
Wiesel experiment, visit https://www.
youtube.com/watch?v=y_l4kQ5wjiw.
CNN architecture
CNNs are the most preferred deep
learning models for image classification
or image related problems. Of late,
CNNs have also been used to handle
problems in fields as diverse as
natural language processing, voice
recognition, action recognition and
document analysis. The most important
characteristic of CNNs is that they
Sample Cell Complex Cell Hypercomplex Cells
Local Receptive Fields
Figure 1: Illustration of local receptive fields and cells in the human visual cortex
automatically learn the important
features without any guided supervision.
Given the images of two classes, for
example, dogs and cats, a CNN will be
able to learn the distinctive features of
dogs and cats by itself.
Compared to other deep learning
models for image related problems,
CNNs are computationally efficient,
which makes them the preferred
option since they can be configured
to run on many devices. Ever since
the arrival of AlexNet in 2012,
researchers have successfully built
new CNN architectures that have
very good accuracy, with powerful
and efficient models. Popular CNN
model architectures include VGG-16,
Inception models, ResNet-50, Xception
and MobileNet.
In general, all CNN models have
architectures that are built using the
following building blocks. Figure 2
depicts the basic architecture of a CNN
model.
ƒƒ Input layer
ƒƒ Convolution layers
ƒƒ Activation functions
ƒƒ Pooling layers
ƒƒ Dropout layers
ƒƒ Fully connected layers
ƒƒ Softmax layer
In the CNN architecture, the
building blocks involving the
convolution layer, activation function,
pooling layer and dropout layer are
repeated before ending with one or
more fully connected layer and the
softmax layer.
Final Object Seen

Activation Function
Input Image Pooling Layer
Convolution Layer
Figure 2: Illustration of CNN model architecture
Kernel
Input Image
Figure 3: Convolution operation
Convolution layer
The convolution layer is the most
important building block of the CNN
architecture. The neurons in the first
convolution layer are not connected to
all the pixels in the input image, but
are connected to pixels of their local
receptive fields, as shown in Figure
2. Mathematically, the convolution
layer merges two sets of information
to produce a feature map. It applies the
convolution filter or kernel to the input
and produces the feature map. Let us
understand this better with an example.
Let us suppose that the input image
size is 6 x 6 and the kernel size is 3
x 3. To draw a parallel to the visual
cortex, the visual field size is 6 x 6
and the local receptive field size is 3
x 3. To generate the feature map, the
kernel is made to slide over each pixel
location of the input image. Hence
a single value is generated for each
convolution operation (the summation
or multiplication of the pixel value
with the corresponding kernel value
Dropout Layer
Softmax Layer
Fully Connected Layers
Feature Map
in the same location). The area on
the input image that is used during
each convolution operation is the
receptive field, the size of which is
equivalent to the kernel size. Since the
kernel is made to slide over the entire
input image, an equal-sized feature
map as the input image is produced
after the convolution operation is
applied to the entire image. The same
procedure is followed for 3D images
or multi-colour channel input images.
An independent feature map is also
produced for each channel in that case.
Since there is one feature map for each
convolution operation using a kernel,
for a normal 2D image in grey scale,
the number of feature maps produced
in each layer is equal to the number
of kernels used at that layer. For 3D
images, it is three times the number of
kernels due to three colour channels.
During a convolution operation, for
each row x and column y in the feature
map, the value at position (x, y) will be
the total of the multiplication of values
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  85
Let's Try Developers
in rows x to:
x + kh – 1, columns y to y + kw – 1
…where kh and kw are the height
and width of the receptive fields. Hence,
to produce a feature map equal to the
input image size, suitable padding has
to be done around the input image. In
general, zero padding is applied to the
input image.
It is also possible to slide the kernel
over the input image by spacing out the
pixel location. That means, as described
above, the kernel is made to slide over
each pixel location of the input image to
generate the feature map. The distance
between each successive receptive field
is one (based on pixel locations). The
spacing out or distance between two
successive receptive fields is called
the stride. As the kernel is made to
slide over rows and columns, the stride
can be different for sliding across
rows (vertical direction) and columns
(horizontal direction). With the stride
value more than one, it is possible to
produce a feature map that is smaller
than the input size. With the stride, the
neuron value in the feature map located
at (x, y) is produced by rows.
(x) * sh + kh - 1, columns (y) * sw +
kw – 1,
…where sh and sw are horizontal
and vertical strides.
Each location (x, y) in the feature
map will be a neuron generated using
the specific kernel in that convolution
layer. The neuron’s weight at a location
(x, y) will hence be a value, after the
convolution operation is applied to
rows x to:
x + kh – 1, columns y to y + kw – 1,
…where kh and kw are the height and
width of the receptive field of the input.
The CNN model is built to learn
the equation WX + b, where W is
the weight of the neurons, X is the
Developers Let's Try
activation values of neurons, and b is
the bias term. All neurons in a feature
map share the same weights (kernel)
and bias term, but different activation
values that correspond to the values
of the receptive field corresponding to
that neuron in the feature map. This is
one of the reasons why computations
are less in a CNN compared to a DNN
(deep neural network). A feature learnt
by a neuron at one particular location
will allow it to detect the same feature
anywhere else in the input, regardless of
the location. This is another important
feature of the CNN as compared to
a DNN, which is not translational
invariant.
Next, consider the convolution
layers after the first convolution layer.
The neuron at location (x, y) in the
feature map k in a convolutional layer l
gets its input from all the neurons in the
previous layer l -1, which are located
in rows:
x * sw to * sh + kw - 1, columns y *
sh to y * sw + kh – 1
…and for all feature maps in the
layer l – 1. These are the same inputs
to all neurons located in the same row
x and column y, in all the feature maps.
This explains how the features learnt
in one layer get integrated to learn
combined features or complex patterns
in successive layers. This also explains
how a CNN learns distinct features
at the initial layers; and convolutes
to integrate basic features to learn
complex distinct features or patterns
in the intermediate convolution layers,
before finally learning objects present
in the input in the last convolution
layers. As discussed earlier, any feature,
pattern or object learnt by the CNN
at any layer will allow it to detect this
anywhere else in the input, independent
of the location. So one needs to be extra
careful when using a CNN to detect
combined objects that are similar in
the input. For example, in the case of
face detection, if one is interested in
86  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
studying the features of the eye and
eyebrows together as a combination,
then normal CNN would learn the
features of the eyebrow and the eye as
two different features, and it may not
help in learning the minor differences
of the combination of features. In that
case, the convolution layers have to
be tweaked to learn those features
appropriately.
Activation feature maps in CNN
Although activations are generated
for each neuron, they may not be
propagated to the final prediction.
Let us now get the equations to
determine the output of a neuron at a
convolutional layer. For a grey scale
image (single channel) the output of
the neuron in the first convolution layer
located at (x, y) of the feature map k is
given by the following:
zx, y = Σm=1 to kh (Σn=1 to kw (ai, j
* wm,n)) + bk for i = m * sh + kh – 1
and j = n * sw – 1 and ai, j
…which will be the pixel values of
the input image. And by:
kh, kw, sh, and sw
 
…which are the height and width
of the kernel and the horizontal and
vertical strides, respectively.
The output of the neuron in any
convolutional layer l (l is not the first
convolutional layer) and for the feature
map k is given by the following:
zx, y, k = Σm=1 to kh (Σn=1 to kw
(Σfm = 1 to kl-1 (ai, j, fm * wm, n,
fm, k))) + bk for i = m * sh + kh – 1
and j = n * sw – 1 and ai, j, fm
…which will be the activation
values of previous layer l – 1. And by:
kh, kw, sh, and sw
…which are the height and width
of the kernel, and the horizontal and
vertical strides, respectively.
zx, y, k
The above is the output of the
neuron located at row m, column n in
the feature map k of the convolutional
layer l.
kl-1
The above is the number of feature
maps in the layer l-1 (previous layer).
ai, j, fm
The above is the activation output of
the neuron located in layer l – 1, row i,
column j, and feature map fm. bk is the
bias term for feature map k (in layer l).
wi, j, fm, k
The above is the weight (kernel)
of any neuron in the feature map k of
the layer l and its input located at row
m, column n, relative to the neuron’s
receptive field and feature map fm.
The feature maps produced in the
convolution layer are input to the next
layer (pooling layer) after applying the
activation function. The feature map
generated after applying the activation
function is called the activation feature
map. The visualisation of the activation
feature map for a filter in a particular
convolution layer depicts the regions of
the input that are activated in the feature
map after applying the filter. In each
activation feature map, the neurons
have a range of activation values with
the maximum value representing the
most activated neuron, the minimum
value representing the least activated
neuron, and the value zero representing
a neuron not activated.
Let us consider an illustration. The
AlexNet model is used for transfer
learning to build a binary classifier
that classifies the given flight image
to either a passenger flight or a fighter
flight. AlexNet has five convolutional

Figure 4: Activation feature maps of a passenger flight
Figure 5: Activation feature maps of fighter flight
layers — the first four convolutional
layers are frozen and the model is
trained to obtain very good accuracy of
around 98 per cent. The figures 4 and 5
show the visualizations of the activation
feature maps of the fifth convolutional
layer, which has 256 filters and hence
256 activation feature maps. Figures 4
and 5 show the following details.
ƒƒ The activation feature maps for
the passenger flight image and
the fighter flight image for the
newconv5 convolution layer
(the fifth convolutional layer
of the AlexNet model, renamed
to newconv5 since it has been
retrained during transfer learning).
ƒƒ The passenger and fighter
flight test images (top left
corner in the figure) for which
visualisations are shown.
ƒƒ Assume the activation feature maps
are arranged as a 16 x 16 matrix
with index 1 given for activation
feature map at row 1, column 1; and
index 17 for the activation feature
map at row 2, column 1, and so on.
ƒƒ The higher resolution image of the
selected activation feature map
(green coloured box) is shown just
below the test image.
In Figure 4, check the 137th
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  87
Let's Try Developers
activation feature map (the green
coloured box), which is shown as a
high resolution image below the test
image. The white coloured pixels in
this image show only those features
that have been learnt by this filter
or, in other words, these are the
activated neurons of the feature map
with a value greater than zero. The
remaining pixel values are zero and
hence black in colour, so neurons
corresponding to those pixels are
not activated. From a comparison
between the activation feature map
and the original image, it can be seen
that the features in the activation
feature map correspond to multiple
windows in the input image. This
can also be justified by checking
the top nine highly activated images
shown at the top right corner in the
figure. The top nine images show the
objects that have multiple window
patterns or features. Refer to http://
yosinski.com/deepvis for more details
on visualisation of the CNN model.
A similar analogy is applicable
to neurons in all the feature maps
shown above.
Now, among the 16 x 16
activation feature maps in the
convolution layer shown in Figures
4 and 5, it can be seen that many
of these maps are not showing any
activations. None of the neurons in
these feature maps are activated.
CNN theory states that each filter
represents distinct feature/s at each
layer, and in these figures, each of
the 256 filters represents features of
the passenger or the fighter flight that
are learnt. If there are no activations,
this means that it does not learn any
feature. Since the model is a binary
classifier for classifying passenger
and fighter flights, each of the filters
in the convolution layer will be
learning either passenger or fighter
features. Hence these feature maps
can be classified into four types –
class 0, class 1, mixed and inactive.
ƒƒ Class 0: In these feature maps, the
Developers Let's Try
features learnt belong to class 0 only.
ƒƒ Class 1: In these feature maps, the
features learnt belong to class 1 only.
ƒƒ Mixed: In these feature maps, the
features learnt belong to both class
0 and class 1.
ƒƒ Inactive: These feature maps do
not learn any features of class 0 or
class 1.
The type of activation feature maps
in each layer can be easily visualized
using the Deepvis tool specified above.
Many a times, complex standard models
are used to build a simple binary as
above, or a model for up to ten class
classifiers using transfer learning.
During those scenarios, as seen above,
many filters may not be required since
they will not be learning any features at
all, and fewer filters will be sufficient to
learn features of all the classes. In these
cases, if the requirement is to retain the
original model architecture for some
reason, then the inactive filters can be
removed to reduce the computations
and the size of the model.
The types of filters will also
help to debug when the test image
is mis-classified. The types of filters
at each layer can be studied for both
the classes during training and during
evaluation. The filters activated can
be compared during training and
evaluation. When making a correct
prediction of the test image, the count
of the type of filters for each class
will approximately match the average
count of the type of filters for each
class during training. This means that
if a test image is correctly classified
as a passenger flight class, then its
count of the type of filters (class 0,
class 1, mixed and inactive) must be
approximately equal to the average
count of the type of filters of the
passenger flight determined during
training, and the same will be true for
an image correctly classified as the
fighter flight class.
These findings can help to debug
mis-classified images. For example, if
the test image is mis-classified as the
88  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
fighter class, then the count of the type This article provides an overview
of filters will not match the count of the of how the visual cortex perceives
type of filters for the fighter class during objects. It explains the parallels
training, which is quite obvious. The between CNNs and different building
potential reason for mis-classification blocks of the CNN architecture. It
could be any of the following: also briefly covers each building
ƒƒ The model has evaluated the block and how the convolution
features of both the fighter and layer processes the image features
passenger flight classes, and found from the input. It then highlights
that fighter features dominating the importance of activation during
the passenger features during image classification. Knowing
classification and hence it is mis- the details about the activations
classified. of each filter in each layer allows
ƒƒ The model has evaluated and found model developers to understand the
more of fighter features matching importance of each filter and update
during classification and hence it is their CNN architectures or build new
mis-classified. CNNs with reduced parameters and
ƒƒ The model has evaluated and computations. The article also gives
found comparable fighter and details on the types of activation
passenger features matching during feature maps, and how they can be
classification and due to minor used for debugging CNN models and
differences in features learnt, it has to detect potential mis-classification.
been mis-classified Further exploration and research on
To understand the above activation feature maps for different
reasoning, the activation feature maps types of models other than CNN will
of each convolution layer need to be provide many insights, especially
analysed, and the average count of for object detection and recognition
the type of filters at each layer may models. The same can be extended to
have to be considered to come to a natural language processing and voice
conclusion. There are approaches to recognition models, the activation
handle this, which will be discussed feature maps for which would be a bit
in the next article. more complex to analyse.
References
[1] https://towardsdatascience.com/applied-deep-learning-part-4-convolutional-
neural-networks-584bc134c1e2
[2] http://yosinski.com/deepvis
[3] https://towardsdatascience.com/wtf-is-image-classification-8e78a8235acb
[4] https://www.tensorflow.org/
By: B.N. Chandrashekar, Dr Manjunath Ramachandra and Shashidhar Soppin
B.N. Chandrashekar is a principal consultant and researcher with hands-on programming
experience. He has around two decades of industry experience. With a Masters from
IISc, he specialises in embedded, retail, e-commerce and AI driven technology.
Dr Manjunath Ramachandra has over two decades of work experience in the
overlapping verticals of artificial intelligence, computer vision, healthcare and wireless/
mobile technologies. He figures in the list of the ‘2000 outstanding intellectuals of the
21st century’, brought out by the International Biographical Center, UK.
Shashidhar Soppin, DMTS (distinguished member of the technical staff) senior
member of Wipro, has 19+ years of experience in the IT industry. He specialises in
virtualisation, Docker, the cloud, AI, ML, deep learning and OpenStack.

Using spaCy for Natural
Language Processing
and Visualisation
spaCy is an open source Python library that lets you break down textual data into machine
friendly tokens. It has a wide array of tools that can be used for cleaning, processing and
visualising text, which helps in natural language processing.
N
atural language processing
(NLP) is an important
precursor to machine
learning (ML) when
textual data is involved. Textual data
is largely unstructured and requires
cleaning, processing and tokenising
before a machine learning model can
be applied to it. Python has a variety
of NLP libraries available for free
such as NLTK, TextBlob and Gensim.
However, spaCy stands out when it
comes to its speed of processing text
and applying beautiful visualisations
to aid in understanding the structure of
text. spaCy is written in Cython; hence
the upper case ‘C’ in its name.
This article discusses some key
features of the spaCy library in action.
By the end of this article, you will
have hands-on experience in using
spaCy to apply all the basic level text
processing techniques in NLP. If you
are looking for a lucrative career in
machine learning and NLP, I would
highly recommend adding spaCy to
your roster.
Installing spaCy
As with most of the machine learning
libraries in Python, installing spaCy
requires a simple pip install command.
While it is recommended that you use a
virtual environment when experimenting
with a new library, for the sake of
simplicity, I am going to install spaCy
without a virtual environment. To install
it, open a terminal and execute the
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  89
Let's Try Developers
following code:
pip install spacy
python -m spacy download en_core_web_sm
spaCy has different models that
you can use. For English, the standard
model is en_core_web_sm. For the
coding part, you will have to select a
Python IDE. I personally like to use
Jupyter Notebook, but you can use any
IDE that you are comfortable with.
Even the built-in Python IDLE works
fine for the code covered in this article.
Once you are on your IDE, test if
the installation has been successful by
running the following code. If you get
no errors, then you are good to go.
import spacy
myspacy = spacy.load(‘en_core_web_sm’)
The myspacy object that we have
created is an instance of the language
model en_core_web_sm. We will use
this instance throughout the article for
performing NLP on text.
Reading and tokenising text
Let’s start off with the basics.
First create some sample text and
then convert it into an object that
can be understood by spaCy. We
will then apply tokenisation to the
text. Tokenisation is an essential
characteristic of NLP as it helps us
in breaking down a piece of text into
separate units. This is very important
Developers Let's Try
for applying functions to the text such
as NER and POS tagging.
#reading and tokenizing text
some_text = “This is some example text
that I will be using to demonstrate the
features of spacy.”
read_text = myspacy(some_text)
print([token.text for token in read_
text])
The following will be the output of
the code:
[‘This’, ‘is’, ‘some’, ‘example’,
‘text’, ‘that’, ‘I’, ‘will’, ‘be’,
‘using’, ‘to’, ‘demonstrate’, ‘the’,
‘features’, ‘of’, ‘spacy’, ‘.’]
We can also read text from a file as
follows:
#reading and tokenizing text from a file
file_name = ‘sample_text.txt’
sample_file = open(file_name).read()
read_text = myspacy(sample_file)
print([token.text for token in read_
text])
Sentence detection
A key feature of NLP libraries is
detecting sentences. By finding the
beginning and end of a sentence, you
can break down text into linguistically
meaningful units, which can be very
important for applying machine
learning models. It also helps you in
applying parts of speech tagging and
named entity recognition. spaCy has
a sents property that can be used for
sentence extraction.
#sentence detection
sample_passage = “This is an example
of a passage. A passage contains many
sentences. Sentences are denoted using
the dot sign. It is important to detect
sentences in nlp.”
read_text = myspacy(sample_passage)
sentences = list(read_text.sents)
for sentence in sentences:
print(sentence)
90  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
As you can see in the following
output, we have successfully broken
down the sample_passage into
discernible sentences.
This is an example of a passage.
A passage contains many sentences.
Sentences are denoted using the dot
sign.
It is important to detect sentences in
nlp.
Removing stop words
An important function of NLP is to
remove stop words from the text. Stop
words are the most commonly repeated
words in a language. In English, words
such as ‘are’, ‘they’, ‘and’, ‘is’, ‘the’,
etc, are some of the common stop
words. You cannot form sentences that
make semantic sense without the usage
of stop words. However, when it comes
to machine learning, it is important
to remove stop words as they tend to
distort the word frequency count, thus
affecting the accuracy of the model.
spaCy has a list of stop words in its
library for English. To be precise, there
are 326 stop words in English. You can
remove them from the text using the
is_stop property of spaCy.
#removing stopwords
print([token.text for token in read_
text if not token.is_stop])
After removing the stop words,
the following will be the output of our
sample text.
[‘example’, ‘passage’, ‘.’,
‘passage’, ‘contains’, ‘sentences’,
‘.’, ‘Sentences’, ‘denoted’, ‘dot’,
‘sign’, ‘.’, ‘important’, ‘detect’,
‘sentences’, ‘nlp’, ‘.’]
Lemmatisation of text
Lemmatisation is the process of
reducing the inflected forms of a word
such that we are left with the root of the
word. For example, ‘characterisation’,
‘characteristic’ and ‘characterise’,
are all inflected forms of the word
‘character’. Here, ‘character’ is the
lemma or the root word. Lemmatisation
is essential for normalising text. We
use the lemma_ property in spaCy to
lemmatise text.
#lemmatisation of text
for word in read_text:
print(word, word.lemma_)
The following is the lemmatised
output of the sample text. We output the
word along with its lemmatised form.
To preserve page space, I am sharing
the output of a single sentence from our
sample text.
Sentences sentence
are be
denoted denote
using use
the
dot dot
sign sign
. .
Finding word frequency
The frequency at which each word
occurs in a text can be vital information
when applying a machine learning
model. It helps us to find the main
topic of discussion in a piece of text
and helps search engines provide users
with relevant information. To find the
frequency of words in our sample text,
we will import the Counter method
from the collections module.
#finding word frequency
from collections import Counter
word_frequency = Counter(read_text)
print(word_frequency)
The following is the output for the
frequency of words in our sample text:
Counter({This: 1, is: 1, an: 1,
example: 1, of: 1, a: 1, passage: 1,
.: 1, A: 1, passage: 1, contains: 1,
many: 1, sentences: 1, .: 1, Sentences:
1, are: 1, denoted: 1, using: 1, the:

1, dot: 1, sign: 1, .: 1, It: 1, is:
1, important: 1, to: 1, detect: 1,
sentences: 1, in: 1, nlp: 1, .: 1})
POS tagging
Parts of speech (POS) tagging helps
us in breaking down a sentence and
understanding what role each word
in the sentence plays. There are eight
parts of speech in the English language,
i.e., noun, pronoun, adjective, verb,
adverb, preposition, conjunction and
interjection.
In POS tagging, we apply a tag to
each word in a sentence that defines
what part of speech that word represents
in the context of a given sentence.
In spaCy, we will make use of two
properties — tag, which gives the fine-
grained part of speech, and pos, which
gives the coarse-grained part of speech.
The spacy.explain() method provides a
description of a given tag.
#POS tagging
for word in read_text:
print(word, word.tag_, word.pos_,
spacy.explain(word.pos_))
The following is the output of the
POS tagging of the sample text. In order
to preserve page space, I am providing
only a portion of the output.
This DT DET determiner
is VBZ AUX auxiliary
an DT DET determiner
example NN NOUN noun
of IN ADP adposition
a DT DET determiner
passage NN NOUN noun
Visualising POS tagging using
displaCy
spaCy comes with a built-in visualiser
called displaCy, using which we can
apply and visualise parts of speech
(POS) tagging and named entity
recognition (NER). To visualise POS
tagging for a sample text, run the
following code:
Let's Try Developers
attr
det
nsubj amod
This is an example sentence.
DET AUX DET NOUN NOUN
Figure 1: POS tagging visualisation using displaCy
Every year DATE , Hyderabad hosts the biggest exhibition in India GPE . It has fun rides and food stalls. At least a million CARDINAL people visit it
Every year DATE .
Figure 2: NER visualisation using displaCy
#using displacy for POS tag least a million people visit it every
visualisation year.”
from spacy import displacy read_text = myspacy(sample_text)
sample_text = “This is an example displacy.serve(read_text, style=”ent”)
sentence.”
read_text = myspacy(sample_text) Figure 2 shows the output of the
displacy.serve(read_text, style=”dep”) NER visualisation.
In this article, we have discussed
Figure 1 shows the output of the the basic functionalities most
code, which you can view by using your commonly used for NLP using
browser on the localhost link provided spaCy and its built-in visualiser
by displaCy. displaCy. There are many more
advanced features in this library that
Visualising NER using displaCy are absolutely worth exploring and
Named entity recognition (NER) is the mastering if you want a solid foundation
process of identifying named entities in in NLP. However, those features are
a piece of text and tagging them with beyond the scope of this article. Hence,
a pre-defined category, such as a name this is a good stopping point. I have
of a person, location, organisation, shared a few links as reference to help
percentage, time, etc. spaCy has the you get a deeper understanding of the
property ents, which we can use to spaCy library. I recommend you go
apply NER on text. We will also use through them.
displaCy to visualise the NER output.
#using displacy for NER visualisation
References
sample_text = “Every year, Hyderabd
[1] https://spacy.io/models
hosts the biggest exhibition in India.
[2] https://spacy.io/usage
It has fun rides and food stalls. At
By: Mir H.S. Quadri
The author is a scholar and researcher in the fields of artificial intelligence and machine
learning. He shares a deep love for Web development and has worked on multiple
projects using a wide array of frameworks. He is also a FOSS enthusiast and actively
contributes to several open source projects. You can find his works on codelatte.site.
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  91
Developers Let's Try

SPA JS: Building Cross-

Platform SPAs with Less Code
SPA JS is a single page application that is simplified and developed using JavaScript.
It is used to create a cross-platform application using which you can develop a Web
app, and the same code can also be used to develop desktop apps. If the code
is bundled with the Electron software, then the result would be a desktop app,
and if you put the code under a Cordova stack, then you would get Android or iOS
applications. So, you can use the same code of SPA JS to develop applications for
the desktop as well as the mobile.

T
he primary aim of SPA JS is
to keep development simple
and eradicate repetitions.
It also follows the crucial
principle, YAGNI (You Ain't Gonna
Need It), which states that you should
“Always implement things only when
you need them but never when you
just think that you may need them.''
For example, in open source software,
contributors may develop different
modules by assuming that you may need
them. However, these modules may not
be useful at all.
Why SPA JS?
Many of you may be familiar with
Angular, React and the Vue.js
framework but very few may be aware
about frameworks like Knockout,
Ember and Backbone. It’s not that
Angular, React, etc, are really game
changers. We already had the concept
of building a single page application
using these frameworks. So, let us
examine the problem with Angular,
React or Vue.js, as well as other
programming options.
Problem 1: The question is why
do we have different frameworks?
This is because developers try to build
some applications on one of these
frameworks, but the latter do not solve
their particular problem in the way they
want to. So, developers try to create
their own frameworks that solve their
individual problem, but this leads to a
lot of frameworks.
Problem 2: Earlier, we had frontend
designers and backend developers.
The former designed the page, while
the latter used programming languages
like C, C++, Java and .NET to build
a page and call APIs. JavaScript was
then introduced as frontends for micro-
interactions, but frontend designers
faced difficulties designing pages with
it. To address this situation, backend
developers were brought to the frontend.
But unfortunately, backend developers
are generally not good in design.
So, companies and developers
are facing the issue of how to create a
perfect bridge between the frontend and
backend. Recently, the term ‘full-stack
developer’ has emerged, describing those
who know all the backend development,

92  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


Figure 1: SPA folder structure
including databases. However, the full-
stack developer doesn't know all the
developments related to every platform
or framework, and that's the current
problem being faced.
Problem 3: Earlier, developers
used to do programming in BASIC or
FORTRAN. For them, the Web was just
CGI or Perl. They used to dump all the
code in a particular Perl file including
HTML, Perl scripts, etc, but it used to
be a nightmare to manage this. Then
developers started using Java, but
even Java servlets were painful. They
wrote HTML code under double quotes,
which was a horrendous task for HTML
developers. So, they needed a separation,
like the MVC framework. But after
the launch of JSP, which is similar to
HTML with the integration of simple JS,
developers are now using JSP with a lot
of JavaScript and less HTML, leading
to complexity of the code. The problem
is that developers seem to be mixing JS,
HTML and even CSS, making things
even more complex.
The first law of software quality
is e=mc² (errors=(more code)²).
This implies that the more the code,
the more the errors. Just take an
example of Angular or React. The
seed project that we download with
one package.json, with one or two
files like index.html, does not allow
us to start our project. We now need
to do a npm install, which downloads
approximately 120MB of code. And all
this needs to be done even if you need
to just demonstrate ‘Hello World’.
We think that we are writing less
code but, unfortunately, we are not — it
is more likely we are writing less code
as someone else has already written it,
and we have installed it along with our
seed project.
Steve Jobs explained the problem
like this, "The way you get programmer
productivity is not by increasing the
lines of code per programming per
day. That doesn't work. The way
you get programmer productivity is
by eliminating the lines of code that
you have to write. The goal here is to
eliminate 80 per cent of the code that
you have to write for your app. That's
the goal. I’ve seen a lot of demos that
try to take it all the way back into the
algorithmic part of the code base, and
none of them have ever been any good."
So, the less the code, the fewer the
errors, and the higher the productivity.
Let us get back to SPA JS
To get started, visit https://spa.js.org/.
The site has all that is needed to
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  93
Let's Try Developers
learn and use SPA JS. Also, it is on a
pure MVC framework that provides
flexibility to developers.
The structure is well organised,
which helps to reduce the code
complexity. Even when the developers
try to mix up the code, SPA JS will
not allow it. It has a clear view that is
separated with three layers — one layer
with pure HTML content, the second
with all the controls, and the data layer
(which is entirely API driven) that helps
to build the perfect bridge. One of the
significant problems with Angular.js
is that the features keep on updating
periodically, and it is challenging to
teach developers these features and
frameworks. SPA JS has tried to address
this issue. The site also has all the
guides, so developers don't need to
Google for a solution.
Disclaimer:
The views presented in the article are
personal and do not represent the views
of the organisation the author works for.
By: Kumar P.
The author is a consulting engineer and
chief frontend architect at Unisys. This
article is based on the talk he presented
during Open Source India 2019.
Admin Insight
A Headless CMS:
Delivering Pure Content in the
Age of Mobile-first Internet
CMS
W
hat started on October
29, 1969, with
ARPANET (under
the US Department of
Defence) delivering a message from
one computer to another, became the
Internet as we know it today. It is the
hub of information that’s available
and accessible around the globe to
everyone. About 60 per cent of the
world’s population has access to the
Internet and there are innumerable
apps, software, distributed systems and
smart devices that use it — enabling
what was once in the realm of sci-fi
movies and stories. The Internet today
is one of the most profitable mediums to
make money. From small time creators
to large Internet companies, everyone
depends on the content to generate
revenue. This is coupled with analytics
to understand what people like,
94  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
A headless CMS (content
management system) is a back-
end-only version built from the
ground up as a content repository
that makes content accessible via
a RESTful API for display on any
device. It is designated ‘headless’
because it does not have a
front-end. A headless CMS is
focused on storing and delivering
structured content.
individually and as part of communities, This gave birth to content
so that the content can be created to suit management systems (CMSs), which
their tastes and mood. revolutionised the way we create
information, store it and present it
The need for CMSs to users. The CMSs enabled people
Much of what you experience as the to collaborate to create content
Internet is powered by the information that and share it with others, in an
is stored in back-end systems of the apps instant. Soon, people didn’t need
that you use today. Internet companies, to understand HTML in order to
schools, universities and even governments create and share content, as the CMS
store information about you to understand evolved to accommodate complex
your behaviour, share content with you content creation requirements and
and to function in meaningful ways. provided the building blocks to enable
When the World Wide Web became collaboration, document management
a phenomenon, it allowed people to and storage.
share information without boundaries. As content became king and with
Anyone with a basic understanding of more users and complex content, there
HTML could create Web pages and was the need to ensure that users could
host them on a website. It was just Web access the content whenever they
pages initially and when things caught wanted and from wherever they were.
on, we needed something better to The content delivery networks (CDN)
create, store and present the content. evolved to meet this need. The CDNs

try to cache the static content as close
as possible to the users so that the
content servers don’t have to serve the
information that doesn’t change much
every time there is a request for it.
Content management systems
As the technology moved forward,
computing moved from large
mainframes to smaller PCs. And the
devices just kept getting smaller — to
mobile phones, smart devices such as
smart watches and digital assistants.
A traditional CMS includes two
major components – a content storage
and creation/management system that
allows a content creator to create,
manage and modify the content and
content types; and a content delivery
system that builds the content and
delivers it to the users when they request
for content. The former provides the
content storage and management, and
the latter is the presentation layer that
fetches the raw content, and applies
styles and the template to compile the
final layout and deliver it to users. It is
also possible to add more functionality
by way of plugins or addons that
complement the content delivery with
more features such as advertising.
This worked when PCs were the
primary devices to consume content.
Think of a blog, video blog or even a
news site – a traditional CMS ensures
that the content writers can push content,
collaborate with each other and then,
the content gets delivered to users in the
right styles and formatting without the
creators doing anything about it.
Pure content management
However, content consumption moved
from large screens to smaller ones —
think mobile phones and smart watches
— and even smart devices and digital
assistants that do not have a screen.
Traditional CMSs needed to now also
provide a way to deliver content on
these new devices and form factors.
This was tackled by adding support
for responsive themes that can adapt to
Database
Figure 1: Basic design of a headless CMS
any screen size and layout. For devices
without a screen, an API layer was
added on top of the content storage
engine. The apps that run on these
devices can call these APIs and deliver
the content as needed.
This works but it’s just only a
matter of ‘when’ rather than ‘if’, the
computing and consumption is due
for a change – just like smartphones
arrived a little over a decade ago and
changed our lives. We are not sure
what this change will be and if you
follow the innovations in hardware
and technology, you are bound to
have heard of things like transparent
displays, foldable phones and even
wall-sized panels.
That’s where pure content
management system comes into the
picture – a system that can pave the
way for any means of delivery on any
kind of device with any size or type of
display – even ones without a display!
Open source developers (and others)
have already done that and these systems
are called headless CMSs – named so
since they do not have a dependency,
or a coupling with a presentation layer
found in the traditional CMSs. Since
headless CMSs store content in a
structured format and without any notion
about how the content will be consumed
or presented to the users, they are
considered ‘pure’. Figure 1 illustrates a
basic design of a headless CMS.
A number of popular headless
CMSs are based on Jamstack (https://
jamstack.org/), which stands for
JavaScript, APIs and mark-up, and
any software that doesn’t have a tight
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  95
Insight Admin
API Clients
coupling between the server and
client qualifies.
However, you will also find
headless CMSs that use frameworks
or languages other than JavaScript.
Jamstack can be used on a variety
of operating systems, Web servers,
databases, back-ends or front-ends. It
provides a higher degree of flexibility,
extensibility and scale, as well as a
great developer experience.
One thing is for sure — a headless
CMS does not include a content-serving
mechanism of any kind and that leads
us to the following question.
How do I deliver content?
The traditional systems deliver content
with their content delivery component.
As a content creator, you are limited by
their capabilities in this area. Additional
engineering is needed to accommodate
any other forms of content delivery that
you may need. This means that you
need to spend time and resources to
ensure all your users can get the content
in the same consistent way.
A headless CMS is meant to be
designed API-first, and it exposes
APIs for you to consume and serve the
content from any medium.
Some of the traditional CMSs, like
WordPress for example, also provide
APIs to bridge this gap. But they are
designed to be all-in-one solutions, so
decoupling the parts you don’t need is
not possible. You can choose to ignore
them though.
That’s a Catch-22 situation – you
either end up wasting computing
resources on components you don’t
Admin Insight
need to run, or you use them as best you
can and fork your code to accommodate
different consumption mediums.
With a headless system, you gain
complete control over how the content
stored in it will be used; and you can
design a pipeline that doesn’t need to be
refactored any time you want to support
a new medium for your users, be it a
smart watch or an AR system.
If you have run into the challenges
mentioned with a traditional CMS,
you are most likely using one of the
workarounds mentioned as well. If not,
you need to start thinking ahead about
when you will run into them and how
you will address them, starting today.
Leveraging a headless CMS
Headless CMSs not only can be used in
places where a CMS is needed but also in
other scenarios where you need to store
content in a structured and consistent
format. Let’s explore this further!
Almost any scenario that you
believe requires a CMS is a candidate
for a headless CMS too. However,
headless CMSs also have uses beyond
their traditional content serving
functions. If you look at the tech
trends of the last couple of years, you
might have realised that chatbots,
smart devices and digital assistants are
topping the charts. These interfaces are
powered by the innovation in hardware,
NUI (natural user interface) and of
course, the content that can be served
in a variety of ways.
A good chatbot is powered by
state-of-art NLP models, and optionally,
with a voice interface that allows it
Dynamic
Content
Static Site
Generators
Headless
CMS
Figure 2: A schematic for GitHub Pages
96  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com
to understand what is being asked
in natural language either using text
or voice inputs. Once the processing
for this task is over, the bot makes
itself useful by surfacing the relevant
content. This is similar for the digital
assistants — in their hardware as well as
software avatars.
The content that you consume
this way is usually small snippets of
information that can be sourced from
emails, calendars, third party services
such as flight tracking and weather, etc.
But other content, such as the witty
responses that you get, are best served
from a headless content. These aren’t
only limited to jokes, etc, but include
meaningful content as well.
As an example, consider a food
ordering or a shopping portal. These
services take a large number of partners
and sellers onto their platform, who list
their services or products. A traditional
approach is to store this content in a
NoSQL DB and fetch the content as the
UI is being rendered. On a high level,
this approach requires that you build an
API wrapper that does the following:
ƒƒ Builds a UI to take onboard the
partners, sellers, items and services
ƒƒ Validates the content before it is
saved in the database with the
right structure
ƒƒ Fetches the content when being
served to the users
Add a chatbot or a digital assistant
(or a skill) to the mix and you can serve
the content on those mediums too if
your APIs are designed efficiently.
With a headless CMS, you are left
with just the work needed for Point 1
CDN Clients
listed above, as a headless CMS provides
first class APIs to interact with the data and
data types so you don’t have to worry about
database handling. You can get started with
how you will integrate these APIs with
your application surfaces, be it a mobile
or Web app, chatbot or a skill for a digital
assistant.
Strapi (https://strapi.io/) is one of
the headless CMSs that I am working
with, and it includes a UI to add content
as well as user roles and permissions
to control access to the content. Ghost
(https://ghost.org/) and Netlify CMS
(https://www.netlifycms.org/) are others
that I have tried out a little bit. You
can look at other options at https://
headlesscms.org/.
A headless CMS also makes an
excellent case for Web apps or websites,
even the ones that are a good mix of
static and dynamic content. You can
front-end your headless CMS with
a static site generator such as Hugo
or Gatsby, and achieve much greater
performance while lowering your
computing costs. Static site generators
also support plugins to fetch data from
dynamic sources such as APIs. This
combination offers a number of benefits
when used in conjunction with a CDN.
Dynamic site acceleration solutions
such as Azure Front Door or CloudFront
add an additional layer of performance
benefits that can further reduce the load
on your content repository. A prime
example of this scenario is GitHub Pages
(https://pages.github.com/) that allows
you to host a static website directly from
your GitHub repo, free of cost. Figure 2
is a schematic for this scenario.
Variations of this architecture can be
useful in other scenarios where serving
content is part of the solution.
Challenges with a headless CMS
When looking at the reference
architecture diagram, it is evident that the
lack of a presentation layer in a headless
CMS introduces the flexibility to use
any front-end. But it also means that you
have another piece of the puzzle that
 Insight Admin

you need to account for when decidingecosystem having been built around needs to be built before you can realise
on whether to opt for a headless CMS.them. For example, the popular CMS the value they bring to the table.
A traditional CMS covers your Web WordPress is estimated to have over a Moving to a headless CMS may not
and mobile bases out-of-the-box with 60 per cent share of the overall CMS be a straightforward activity and will
themes to customise the look and feel.
market and 35 per cent share of all require planning and effort to transition
To get started with content delivery on
websites functioning today. One of your workflows and users seamlessly.
a headless CMS, developers will need the reasons is the extensibility and Before you start planning on this, take
to build a user experience (UX) for the
the plugin marketplace this CMS a step back and see if your existing
content. This is also true for the content
offers to customise it from a simple CMS satisfies your current and future
creators – a traditional CMS includesblog to a full-fledged e-commerce content delivery requirements. If you see
content creation and collaboration website. With responsive themes and yourself extending your CMS because
workflows which are non-existent plugins, you can convert WordPress it does not support certain delivery
with the other choice. A pure contentinstallations to PWAs and push them surfaces that either your business plans
management system will also be a into the App Store and Play Store to dictate you use or you believe your users
culture shock to content authors since
provide a more engaged experience are moving to, it is time to consider
well-known concepts such as pages andfor the users. Drupal is another evaluating a headless CMS.
other UI constructs do not exist at all,
contender in this domain.
and a UX for authors will need to be Headless CMSs are a newer breed
built from scratch too. in the content management space, and By: Ashish Sahu
Visit ‘Online Only’ expo-cum-conference
What’s New In Electronics & IoT
they do tick all the boxes for building from
The the comfort
author ofsolutions
is a cloud your HOME…architect
So, is it a traditional CMS scalable and extensible content delivery working with Microsoft India. He helps
or a headless one? pipelines that can support virtually ISVs and startups overcome technical
During These Challenging Times?
Traditional CMSs have been around any kind of device available today and
challenges, adopt the latest technologies
and take their solutions to the next level.
for a long time, with quite an tomorrow. The user experience, though,
VISIT NOW https://IndiaTechnologyWeek.com JUNE EDITION: 17th – 19th JUNE, 2020

What’s New In Electronics & IoT

During These Challenging Times?
Visit ‘Online Only’ expo-cum-conference from the comfort of your HOME…

JUNE EDITION:
17th – 19th JUNE, 2020

VISIT NOW https://IndiaTechnologyWeek.com

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  97

Admin Let’s Try

The Benefits of Using

Terraform as a Tool for
Infrastructure-as-Code (IaC)
DevOps tools have enabled software engineers to deploy application source code in a
better way. Including Infrastructure-as-Code (IaC) in the DevOps cycle helps the transition
to the cloud model, accommodating the shift from ‘static’ to ‘dynamic’ infrastructure.

T
he advent of DevOps helped
minimise the dependence on
sysadmins who used to set
up infrastructure manually,
while seated at some unknown corner
of the office. Managing servers
and services manually is not a very
complicated task in a data centre. But
when we move to the cloud, scale and
start working with many resources
from multiple providers (AWS, GCP,
Azure, etc), manually setting up and
configuring to achieve on-demand
capacity slows things down. Being
repetitive, the manual process is most
likely error-prone. And it cannot be
managed and automated when working
with resources from different service
providers together.
Infrastructure-as-Code (IaC)
This approach is the management
and configuration of infrastructure
(virtual machines, databases, load
balancers and connection topology) in
a descriptive cloud operating model.
Infrastructure can be maintained just
like application source code under
the same version control. This lets
the engineers maintain, review, test,
modify and reuse their infrastructure
and avoid direct dependence on the
IT team. Systems can be deployed,
managed and delivered fast, and
automatically, through the IaC.
There are many tools available
for IaC such as CloudFormation
(only for AWS), Terraform,
Chef, Ansible and Puppet.
What is Terraform?
Terraform is an open source
provisioning tool from HashiCorp
(more can be read at http://terraform.
io/) written in the Go language. It
is used for building, changing and
versioning infrastructure, safely and
efficiently. Provisioning tools are
responsible for the creation of server
and associated services rather than
configuration management (installation
and management of software) on
existing servers. Terraform acts as a
provisioner, and focuses on the higher
abstraction level of setting up the
servers and associated services.
The infrastructure Terraform can
manage includes low level components
such as compute instances, storage,

98  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


and networking, as well as high level
components such as DNS entries, SaaS
features, etc. It leaves the configuration
management (CM) to tools such as
Chef that do the job better. It lays
the foundation for automation in
infrastructure (both cloud and on-
premise) using IaC. Its governance
policy makes the cloud operating model
system-compliant, which otherwise is
only known internally to the IT team.
Terraform is cloud-agnostic and uses
a high level declarative style language
called HashiCorp Configuration
Language (HCL) for defining
infrastructure in ‘simple for humans to
read’ configuration files. Organisations
can use public templates and can have
a unique private registry. Templates
are a maintained repository containing
pre-made modules for infrastructure
components needed under version
control systems (like Git).
Installation
Installing Terraform is very simple; just
follow the steps mentioned below.
1. Download the archive
(https://releases.hashicorp.com/
terraform/${VER}/terraform_${VER}_linux_
amd64.zip):
export VER=”0.12.9”
wget
https://releases.hashicorp.com/
terraform/${VER}/terraform_${VER}_linux_
amd64.zip
2. Once downloaded, extract the
archive:
unzip terraform_${VER}_linux_amd64.zip
3. The last step created a Terraform
bin file in the working directory.
For Terraform to be accessible
everywhere, move it to usr/local/bin:
sudo mv terraform /usr/local/bin
4. Confirm the Terraform installation:
terraform –v //v0.12.9
Life cycle
Terraform template (code) is written
in HCL language and stored as a
configuration file with a .tf extension.
HCL is a declarative language, which
means our goal is just to provide the
end state of the infrastructure and
Terraform will figure out how to create
it. Terraform can be used to create and
manage infrastructure across all major
cloud platforms. These platforms are
referred to as ‘providers’ in Terraform
jargon, and cover AWS, Google Cloud,
Azure, Digital Ocean, Open Stack and
many others.
Let us now discuss each stage in
the IaC lifecycle (Figure 1), which is
managed using Terraform templates.
Code
Figure 2 is sample code for starting
an EC2 or t2.micro instance on AWS.
As visible, only a few lines will
instantiate an instance on AWS. It’s
also implicit that the same code can be
maintained under VCS and be used to
instantiate instances in various regions
with different resource configurations,
removing error-prone and time-
consuming manual work.
ƒƒ Provider: All major cloud players
such as AWS, Azure, GCP and
OpenStack have their APIs for
Figure 1: Life cycle of IaC using Terraform
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  99
Let’s Try Admin
Terraform. These APIs are
maintained by the community.
• Username: Key given by
provider
• Password: Key given by
provider
• Region: Specify region of
deployment
ƒƒ Resource: There are many kinds
of resources such as an open-stack
basic instance, AWS EC2, Droplet
and Azure VM, which can be
created as follows:
• resource <provider_instance_
type> <identifier>
• Image ID: This is machine
image specific, a tag for an
image we need to install
(Ubuntu and Windows).
• Flavour type: This is the
type of instance governing
the CPU, memory and disk
space.
Here, template defines the
provider as AWS, and provides the
access key, secret key and region for
being able to connect to AWS. After
that, resource to be create is specified
ie. an aws_instance here and is
named as “example”. Also, the count
and instance type (size of instance)
is mentioned as code and can be
seen in Figure 2.
The same code can be used to
set up an instance in another region.
Also, Terraform offers users the power
of using variables and other logical
statements such as if-else and the for
loop, which can optimise the setting up
of infrastructure even more.
Figure 2: Terraform template code for
provisioning an EC2 instance
Admin Let’s Try

Maintain and reuse

Configuration templates, i.e., pre-made
modules for infrastructure components
are written, reviewed, managed, and
reused after storing them under VCS.
Organisations can use public templates
contributed by the community as well
as have unique private templates stored
in a central registry.

Init
The Terraform binary contains the basic
functionality and everything else is
downloaded as and when required. The
‘terraform init’ step analyses the code, Figure 3: The ‘Terraform init’ step initialises all the required resources and plugins
figures out the provider and downloads
all the plugins (code) needed by the
provider (here, it’s AWS). Provider
plugins are responsible for interacting
over APIs provided by the cloud
platforms using the corresponding CLI
tools. They are responsible for the life
cycle of the resource, i.e., create, read,
update and delete. Figure 3 shows
the checking and downloading of the
provider ‘aws’ plugins after scanning
the configuration file.

Plan
The ‘terraform plan’ is a dry run for
our changes. It builds the topology of
all the resources and services needed
and, in parallel, handles the creation of
dependent and non-dependent resources.
It efficiently analyses the previously
running state and resources, using a
resource graph to calculate the required
modifications. It provides the flexibility Figure 4: ‘Terraform plan’ dry runs the instantiation
of validating and scanning infrastructure
resources before provisioning, which
otherwise would have been risky. The
‘+’ symbol signifies the new resources
that will be added to the already
existing ones, if any. Figure 4 shows the
generation of the plan and shows the
changes in resources, with ‘+’ indicating
addition and ‘-’ indicating deletion.

Validation
Administrators can validate and
approve significant changes produced
in ‘terraform planning’s’ dry run. This Figure 5: ‘Terraform apply’ instantiates the validated infrastructure in the planning step

100  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com


completely prevents specific workspaces
from exceeding predetermined thresholds,
lessening the costs and increasing
productivity. Also, any standards or
governing policy for the cloud operating
model have not been put in place yet.
A policy has not been codified yet;
rather, certain practices are internally
known among teams and organisations.
Terraform enforces sentinel policies as
code before provisioning the workflow
to minimise the risks through active
policy enforcement.
Apply
The ‘terraform apply’ executes the
exact same provisioning plan defined
in the last step, after being reviewed.
Terraform transforms configuration
files (.tf) based on appropriate API
calls to cloud provider(s) automating
resource creation (using the provider’s
CLI) seamlessly. This will create the
resources (here, an EC2 server) on AWS
in a flexible and straightforward manner.
Figure 5 shows the resources that will
be created, and Figure 6 confirms the
changes that took place.
If we proceed to the AWS console
to verify the instantiation, a new EC2
instance will be up and running, as
shown in Figure 7.
Destroy
After resources are created, there may be
a need to terminate them. As Terraform
tracks all the resources, terminating
them is also simple. All that is needed
is to run ‘terraform destroy’. Again,
Terraform will evaluate the changes and
execute them after you give permission.
Figure 7: EC2 instance can be seen in the initialising state
Additional features of Terraform
1. It provides a GUI to manage all the
running services. It also provides
an access control model based
on the organisation, teams and
users. Its audit logging emits logs
whenever a change (here, change
signifies sensitive write to existing
IaC) happens in the infrastructure.
2. Existing pipeline integration:
Terraform can be triggered from
within most continuous integration/
continuous deployment (CI/CD)
DevOps pipelines such as Travis,
Circle, Jenkins and Gitlab. This
enables a plugin provisioning
workflow and sentinel policies into
the CI/CD pipeline.
3. Terraform supports many providers
(more at https://www.terraform.
io/docs/providers/index.html),
allowing users to easily manage
resources no matter where they
are located. Instances can be
provisioned on cloud platforms
such as AWS, Azure, GCP and
OpenStack using APIs provided by
the cloud service providers.
Figure 6: Terraform showing the absolute changes made to the infrastructure
www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  101
Let’s Try Admin
4. Terraform uses a declarative style
when the desired end state is written
directly. Here, the tool is responsible
for figuring out and achieving that
end state by itself.
Let’s say, we want to deploy five
Elastic instances on AWS using Chef
(Figure 8a) and Terraform (Figure 8b).
Observing the script given in Figure
8, one can see that both are equivalent
and will produce the same results.
But let’s assume a festive sale is
coming up. The expected traffic will
increase, and infrastructure must scale
our application. Let’s say, five more
instances are required to handle the
predicted traffic.
As language is procedural, setting
count as 10 will start additional 10
instances rather than adding extra five, thus
initiating a total of 15 instances. We must
manually remember the previous count, as
shown in Figure 9a. Hence, we must write
a completely new script adding one more
redundant syntax code file.
As language is declarative, setting
the count as 10 (as can be seen in
Figure 9b) will start an additional five
Admin Let’s Try

instances. We do not have to manually

remember the previous count. We have
provided the end state and everything
else will be handled by Terraform
itself. This demonstrates all possible
scenarios when manual interventions
occur, such as slow speed, prone to
human error, etc. Figure 8a and 8b: Sample code for instantiating five instances

Advantages of incorporating
IaC using Terraform
1. Prevents configuration drift:
Terraform, being provisioning
software, binds you to make
changes in your container and only
then deploy the new ones across Figure 9a and 9b: Sample code for instantiating ten instances
every server. This separates server
configuration from any dependency, store local variables such as cloud saving extra infrastructure costs and
resulting in identical instances tokens and passwords in encrypted other overheads.
across our infrastructures. form on the terraform registry. Terraform is an open source tool
2. Easy collaboration: The terraform 5. Masterless: Terraform is masterless, that helps teams manage infrastructure
registry (Terraform’s central registry by default, i.e., it does not need in an efficient, automated and reusable
version control) enables teams to a master node to keep track of manner. It has a simple modular syntax
collaborate on infrastructure. all configuration and distributing and supports multi-cloud infrastructure
3. No separate documentation updates. This saves the extra configuration. Enterprises can use
needed: The code written for infrastructure and maintenance costs Terraform in their DevOps methodology
infrastructure will become your we’d have to incur in maintaining an to construct, modify, manage and deliver
documentation. By looking at the extra master node. Terraform directly infrastructure at a faster pace with less
script, thanks to its procedural nature, uses the cloud providers’ API, thus manual intervention.
we can figure out what’s currently
deployed and its configuration.
By: Vaibhav Aggarwal and Prof. B. Thangaraju
4. Flexibility: Terraform not only
The authors are associated with the open source technology lab in the International
handles IaaS (AWS, Azure, etc) but Institute of Information Technology, Bengaluru.
also PaaS (SQL, NodeJS). It can also

OSFY Magazine Attractions During 2020-21

MONTH THEME
March 2020 Mobile/Web App Development and Optimisation
April 2020 Cybersecurity, Open Source Firewall, Network Security and Monitoring
May 2020 AI, Deep Learning and Machine Learning
June 2020 DevOps Special
July 2020 Blockchain and Open Source
August 2020 Database Management and Optimisation
September 2020 Open Source and IoT
October 2020 Cloud Special: BigData, Hadoop, PaaS, SaaS, Iaas and Cloud
November 2020 Open Source on Windows
December 2020 Open Source Programming (Languages and Tools)
January 2021 Data Security, Storage and Backup
February 2021 Best in the World of Open Source (Tools and Services)

102  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Let’s Try OpenGuru

Lighttpd: A Lightweight
HTTP Server for
Embedded Systems
This article guides readers through the implementation of Lighttpd, a lightweight
interactive HTTP server for embedded systems that have limited memory and storage but
require real-time performance. Lighttpd is also very useful on Linux desktops.

L
ighttpd is an open source Web server optimised
for environments in which speed and high
performance are critical. It is a viable alternative
to Apache or other heavyweight Web servers.
Lighttpd is standards-compliant and has built-in security as
well as flexibility. The entire source code has been written
in C. This comes under the BSD ‘three-clause’ licence. It
also supports fast CGI (Common Gateway Interface) for
creating dynamic content.
Prerequisites
For the system: The desktop/embedded system should run
Linux. The demonstration here is based on an Ubuntu 16.04
32-bit desktop.
For the reader: The reader should know the basics of
HTTP GET/POST/PUT and HTML syntax.
Installation
As the focus is on embedded platforms, the installation
shown is from source code.
To start the installation, you can download the source
code from https://download.lighttpd.net/lighttpd/releases-
1.4.x/lighttpd-1.4.55.tar.gz. You can download it manually
or by using the following command:
$ wget https://download.lighttpd.net/lighttpd/releases-
1.4.x/lighttpd-1.4.55.tar.gz
Next, use the tar command to untar the tarball:
$ tar -zxvf lighttpd-1.4.55.tar.gz
$ cd lighttpd-1.4.55
After extracting the contents from the tarball, we move
to the directory and do the compiling followed by the
installation.
$ ./configure
Note: When you are cross-compiling, you have to
mention the build, host, target and other arguments to
make use of your cross-compiler.
$ CC=”/path/to/cross-compiler-gcc”
$ LD=”/path/to/cross-cmpiler-gnu-ld”
$ ./configure --host=ppc-linux-gnu \
--build=i686-redhat-linux-gnu \
--target=powerpc-*-elf --includedir=/path/to/sysroot-for-
cross
To get help on all the options, type the following
command:
$ ./configure –help

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  103

OpenGuru Let’s Try

Then, to compile and install, use the command given server.errorlog = “/var/log/lighttpd/error.log”
below: accesslog.filename = “/var/log/lighttpd/access.log”

$ make server.modules = (
$ sudo make install “mod_access”,
“mod_accesslog”,
“mod_fastcgi”,
Note: If the target is an embedded one like PPC or
ARM, ‘make install’ is not of any use. Hence you have “mod_rewrite”,
to identify the necessary executables and library after “mod_auth”
compilation, and place them in your file system. )

Once we have successfully compiled and installed the # mimetype mapping

software, create a lighttpd user and group: mimetype.assign = (
“.pdf” => “application/pdf”,
# groupadd lighttpd “.tar.gz” => “application/x-tgz”,
# useradd -g lighttpd -d /home/sganguly/Documents/myserver/ “.tgz” => “application/x-tgz”,
-s /sbin/nologin Lighttpd “.tar” => “application/x-tar”,
“.zip” => “application/zip”,
Here, /home/sganguly/Documents/myserver/ is the root “.mp3” => “audio/mpeg”,
directory for the server. “.css” => “text/css”,
Next, create a log directory for the server to store logs: “.html” => “text/html”,
“.htm” => “text/html”,
# mkdir /var/log/lighttpd “.js” => “text/javascript”,
# chown lighttpd:lighttpd /var/log/Lighttpd “.c” => “text/plain”,
“.conf” => “text/plain”,
Now, create a directory for the documents that the HTTP “.text” => “text/plain”,
server will be serving (Document root): “.txt” => “text/plain”,
“.xml” => “text/xml”,
# mkdir /home/sganguly/Documents/myserver “.mpeg” => “video/mpeg”,

This can be anything of your choice as long as you )

mention it properly in the configuration file. Now create
Index.html file with any content, and place it in this index-file.names = ( “index.html” )
directory.
We are now ready to run the Lighttpd server. The Lighttpd server will start at 127.0.0.1:80 and, by
default, it will load index.html. So, go to a browser and type
# /usr/local/sbin/lighttpd -f /home/sganguly/Documents/ the following command in the browser window:
lighttpd.conf
http//127.0.0.1:80
In the above command, /home/sganguly/Documents/
lighttpd.conf is the configuration file used. The configuration Then hit Enter, and you will be shown the index.html page.
file’s content should be as follows:
Adding support for fast CGI
server.document-root = “/home/sganguly/Documents/myserver” To generate dynamic behaviour and server side computation,
there are PHP, JSP and many other things. But for an
server.port = 80 embedded system with limited memory, in order to get fast
server.username = “lighttpd” responses, C is still the best choice. CGI will allow your C
server.groupname = “lighttpd” code in the server to talk to the HTTP client.
server.bind = “127.0.0.1” Step 1: Download fcgi2 from https://github.com/
server.tag =”lighttpd” FastCGI-Archives/fcgi2.
Step 2: Compile and install.

104  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com

 Let’s Try OpenGuru

int main (void) {

Web Server unsigned char val = 0;
Web Client
(lighttpd)
(browser) HTTP char *env;
Fast CGI

while(FCGI_Accept() >= 0){

CGI binary database
printf(“Status: 200 OK\r\n”);
printf(“Content-type: text/html\r\n\r\n”);
Figure 1: Web server with CGI architecture printf(“<!doctype><html>”);
printf(“<head><meta http-equiv=\”refresh\”
$ sudo apt-get install autogen autoconf libtool /*these are content=\”3\”</head> <body>”);
supporting programs in host PC*/ printf(“val %d<br>”, val);
$ cd fcgi2-master
$ ./autogen.sh env=getenv(“HTTP_USER_AGENT”);
$ ./configure if(env != NULL) {
printf(“User Agent = %s<br>”,env);
}
Note: As before, CC and LD variables have to be set

and the necessary options can be given for embedded
env=getenv(“CONTENT_TYPE”);
cross targets.
if(env != NULL) {
printf(“Content Type = %s<br>”,env);
$ make }
$ sudo make install
printf(“</body></html>\n”);
val++;
Note: As before, collect the binaries and place them }
in a file system for cross targets.
return EXIT_SUCCESS;
Step 3: Now we are ready to write and execute our first }
CGI program. But before starting, edit lighttpd.conf for
linking the CGI binary. FCGI_Accept() will be true with any GET or POST
request. Lighttpd also sets some environment variables to
fastcgi.debug = 1 be accessible from the CGI code.
fastcgi.server = ( Step 4: Let’s compile this code:
“/hellotest” => ((
#”host” => “127.0.0.1”, sganguly@breakout:~/Documents$ gcc hello_cgitest.c -o hello_
#”port” => 51000, cgitest.fcgi -lfcgi -Wl,-rpath /usr/local/lib
“bin-path” => “/home/sganguly/Documents/hello_cgitest. /usr/local/lib is the location where fcgi library is located
fcgi”, after installation.
“socket” => “/tmp/hello_cgitest.sock”,
“check-local” => “disable”, Once you open the page from the browser, it will look
“max-procs” => 2, like what’s shown in Figure 2.
))
)

Here hello_cgitest.fcgi is the name of the binary val 15

User Agent = Mozilla/5.0 (X11; Linux i686; rv;75.0) Gecko/20100101 Firefox/75.0
executable. Socket will be used between the HTTP server Number1:
11
and the CGI binary. To get this page, type 127.0.0.1:80/ Number2:
hellotest in a browser. -5

Submit
#include <fcgi_stdio.h>
Sum = 6
#include <stdlib.h>
#include <string.h> Figure 2: A sample Web page with form (as per code)

www.OpenSourceForU.com  |  OPEN SOURCE FOR YOU  | JUNE 2020  |  105

OpenGuru Let’s Try

For HTTP POST, the entire form’s contents come int tokenNo=0;
as a big string, which can be read and parsed in C code, int inputlen = atoi(getenv(“CONTENT_LENGTH”));
as shown below: fread(buffer, inputlen, 1, stdin);
/*parse input fno=4&ln0=-2 */
int num1,num2,sum; token = strtok(buffer, delim);
printf(“<!doctype> \ while(token!= NULL)
<html> <body> <form method=\”post\” action=\”\”> \ {
<label for=\”fno\”>Number1:</label><br> \ tokenNo++;
<input type=\”number\” id=\”fno\” name=\”fno\” if(2 == tokenNo) num1=atoi(token);
value=%d><br> \ if(4 == tokenNo) num2=atoi(token);
<label for=\”lno\”>Number2:</label><br> \ token = strtok (NULL, delim);
<input type=\”number\” id=\”ln0\” name=\”ln0\” }
value=%d><br><br> \ sum = num1+num2;
<input type=\”submit\” value=\”Submit\”> \ }
</form> \
Sum = %d\ You can see that the form is also generated in the code
</body> \ using the printf statement.
</html><br>”,num1,num2,sum); The entire source code is available at https://github.com/
env=getenv(“REQUEST_METHOD”); SupriyoGanguly/tryLighty.
if(strcmp(env, “POST”) == 0)
{ By: Supriyo Ganguly
char buffer[200] = {‘\0’};
The author is a senior technical officer at Electronics
const char delim[] = “&=”; Corporation of India Limited, Hyderabad.
char *token;

106  |  JUNE 2020  |  OPEN SOURCE FOR YOU  |  www.OpenSourceForU.com