• A common language (for the two ends to understand)
• A way to address who do you want to communicate with
• A Connection (so the content of of the communication can reach the recipients)

A set of protocols that define how two or more devices can communicate with each other. To learn more about TCP/IP, read here

Ethernet simply refers to the most common type of Local Area Network (LAN) used today. A LAN—in contrast to a WAN (Wide Area Network), which spans a larger geographical area—is a connected network of computers in a small area, like your office, college campus, or even home.

When a device sends a packet to the broadcast MAC address (FF:FF:FF:FF:FF:FF​), it is delivered to all stations on the local network. Ethernet broadcasts are used to resolve IP addresses to MAC addresses (by ARP) at the datalink layer .

An Internet Protocol address (IP address) is a numerical label assigned to each device connected to a computer network that uses the Internet Protocol for communication.An IP address serves two main functions: host or network interface identification and location addressing.

You can read more about the OSI model in penguintutor.com

Multicast: Sending a message to a group of subscribers. It can be one-to-many or many-to-many.

1. Before sending a frame, it checks whether another host already transmitting a frame.
2. If no one transmitting, it starts transmitting the frame.
3. If two hosts transmitted at the same time, we have a collision.
4. Both hosts stop sending the frame and they send to everyone a 'jam signal' notifying everyone that a collision occurred
5. They are waiting for a random time before sending again
6. Once each host waited for a random time, they try to send the frame again and so the

roxy servers provide varying levels of functionality, security, and privacy depending on your use case, needs, or company policy.

• A client node sends a SYN data packet over an IP network to a server on the same or an external network. The objective of this packet is to ask/infer if the server is open for new connections.
• The target server must have open ports that can accept and initiate new connections. When the server receives the SYN packet from the client node, it responds and returns a confirmation receipt – the ACK packet or SYN/ACK packet.
• The client node receives the SYN/ACK from the server and responds with an ACK packet.

Bonus question: what is the RTT of LAN?

Penguintutor.com provides a good explanation.

A default gateway serves as an access point or IP router that a networked computer uses to send information to a computer in another network or the internet.

Systems keep an ARP look-up table where they store information about what IP addresses are associated with what MAC addresses. When trying to send a packet to an IP address, the system will first consult this table to see if it already knows the MAC address. If there is a value cached, ARP is not used.

Read more here

NAT stands for network address translation. It’s a way to map multiple local private addresses to a public one before transferring the information. Organizations that want multiple devices to employ a single IP address use NAT, as do most home routers. For example, your computer's private IP could be 192.168.1.100, but your router maps the traffic to it's public IP (e.g. 1.1.1.1). Any device on the internet would see the traffic coming from your public IP (1.1.1.1) instead of your private IP (192.168.1.100).

• SSH - 22
• SMTP - 25
• HTTP - 80
• DNS - 53
• HTTPS - 443
• FTP - 21
• SFTP - 22

The control plane is the part of the network that decides how to route and forward packets to a different location.

The data plane is the part of the network that actually forwards the data/packets.

Refers to monitoring and management functions.

Control Plane.

Latency. To have a good latency, a search query should be forwarded to the closest datacenter.

Throughput. To have a good throughput, the upload stream should be routed to an underutilized link.

• Keep caches updated (which means the request could be forwarded not to the closest datacenter)

00110011110100011101

• Hypertext Transfer Protocol (HTTP) - used for the webpages on the internet
• Simple Mail Transfer Protocol (SMTP) - email transmission
• Telecommunications Network - (TELNET) - terminal emulation to allow client access to telnet server
• File Transfer Protocol (FTP) - facilitates transfer of files between any two machines
• Domain Name System (DNS) - domain name translation
• Dynamic Host Configuration Protocol (DHCP) - allocates IP addresses, subnet masks and gateways to hosts
• Simple Network Management Protocol (SNMP) - gathers data of devices on the network

• Internet Protocol (IP) - assists in routing packets from one machine to another
• Internet Control Message Protocol (ICMP) - lets one know what is going such as error messages and debugging information

Read more [here](https://www.globalsign.com/en/blog/what-is-hsts-and-how-do-i-use-it#:~:text=HTTP%20Strict%20Transport%20Security%20(HSTS,and%20back%20to%20the%20browser.)

The internet refers to network of networks, transferring huge amounts of data around the globe.
The World Wide Web is an application running on millions of server, on top of the internet, accessed through what is know as the web browser

ISP (Internet Service Provider) is the local internet company provider.

• Administration
• Troubleshooting & Debugging
• Storage
• Networking
• Development
• Deployments

• touch - update file's timestamp. More commonly used for creating files
• ls - listing files and directories
• rm - remove files and directories
• cat - create, view and concatenate files
• cp - copy files and directories
• mkdir - create directories
• pwd - print current working directory (= at what path the user currently located)
• cd - change directory

• cd / -> change to the root directory
• cd ~ -> change to your home directory
• cd -> change to your home directory
• cd .. -> change to the directory above your current i.e parent directory
• cd . -> change to the directory you currently in
• cd - -> change to the last visited path

To list all the files, one can run ls -R /dir1

• file permissions, number of links, owner name, owner group, file size, timestamp of last modification and directory/file name

These are files directly not displayed after performing a standard ls direct listing. An example of these files are .bashrc which are used to execute some scripts. Some also store configuration about services on your host like .KUBECONFIG. The command used to list them is, ls -a

myProgram < input.txt > executionOutput.txt

• sed: a stream editor. Can be used for various purposes like replacing a word in a file: sed -i s/salad/burger/g
• grep: a search tool. Used to search, count or match a text in a file:
  • searching for any line that contains a word in a file: grep 'word' file.md
  • or displaying the total number of times a string appears in a file: grep -c 'This is a string' file.md
• cut: a tool for cutting out selected portions of each line of a file:
  • syntax: cut OPTION [FILE]
    • cutting first two bytes from a word in a file: cut -b 1-2 file.md, output: wo

Using the mv command.

• rm -rf dir
• cat or less
• chmod 777 /tmp/x
• cd ~
• sed -i s/good/great/g /tmp/y

• whereis
• which

echo hello world
echo "hello world"

The echo command receives two separate arguments in the first execution and in the second execution it gets one argument which is the string "hello world". The output will be the same.

Using a pipe in Linux, allows you to send the output of one command to the input of another command. For example: cat /etc/services | wc -l

history command or .bash_history file

Most likely the default/generated $PATH was somehow modified or overridden thus not containing /bin/ where df would normally go. This issue could also happen if bash_profile or any configuration file of your interpreter was wrongly modified, causing erratics behaviours. You would solve this by fixing your $PATH variable:

As to fix it there are several options:

Note: There are many ways of getting errors like this: if bash_profile or any configuration file of your interpreter was wrongly modified; causing erratics behaviours, permissions issues, bad compiled software (if you compiled it by yourself)... there is no answer that will be true 100% of the time.

Alternatively if you are using a distro with systemd it's recommended to use systemd timers.

ls > ls_output.txt

yippiekaiyay 2> ls_output.txt

yippiekaiyay 1>&2

An output similar to: yippikaiyay: command not found...
The file die_hard will not be created

The root of the filesystem. The beginning of the tree.

• binaries
• configuration files
• home directories of the different users
• files that tend to change and be modified like logs
• temporary files

/tmp folder is cleaned automatically, usually upon reboot.

False. No one can create file in /proc directly (certain operations can lead to files being created in /proc by the kernel).

The command passed to the boot loader to run the kernel

Using the chmod command.

• setuid is a linux file permission that permits a user to run a file or program with the permissions of the owner of that file. This is possible by elevation of current user privileges.
• setgid is a process when executed will run as the group that owns the file.

• chmod - changes access permissions to files system objects
• chown - changes the owner of file system files and directories
• chgrp - changes the group associated with a file system object

True

• No more disk space
• No more inodes
• No permissions

If we visualize the unix/linux system in layers, systemd would fall directly after the linux kernel.
Hardware -> Kernel -> Daemons, System Libraries, Server Display.

To start a service: systemctl start <service name> To stop a service: systemctl stop <service name>

systemctl status <service name>

journalctl

/var/log

tail -f <file_name>

dstat -t is great for identifying network and disk issues. netstat -tnlaup can be used to see which processes are running on which ports. lsof -i -P can be used for the same purpose as netstat. ngrep -d any metafilter for matching regex against payloads of packets. tcpdump for capturing packets wireshark same concept as tcpdump but with GUI (optional).

dstat -t is great for identifying network and disk issues. opensnoop can be used to see which files are being opened on the system (in real time).

strace is great for understanding what your program does. It prints every system call your program executed.

top will show you how much CPU percentage each process consumes perf is a great choice for sampling profiler and in general, figuring out what your CPU cycles are "wasted" on flamegraphs is great for CPU consumption visualization (http://www.brendangregg.com/flamegraphs.html)

• Check with top for anything unusual
• Run dstat -t to check if it's related to disk or network.
• Check if it's network related with sar
• Check I/O stats with iostat

• Allocating memory
• Schedule processes
• Control CPU

uname -a command

Applications can access system resources and indirectly the kernel space by making what is called "system calls".

• Build time (when it's compiled)
• Boot time (when it starts)
• Runtime (once it's already running)

Usually it will reside in /boot/config-<kernel version>.<os release>.<arch>

/proc/cmdline

sysctl -a

Yes, you might notice that in most systems, when running systctl -a with root, you'll get more runtime parameters compared to executing the same command with a regular user.

Another way to is to run echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward

In the past it was done with sysctl system call, but it was deprecated at some point.

There is a service called systemd-sysctl that takes the content of /etc/sysctl.conf and applies it. This is how changes persist, even after reboot, when they are written in /etc/sysctl.conf

No. Containers have their own /proc filesystem so any change to kernel parameters inside a container, are not affecting the host or other containers running on that host.

An SSH server will have SSH daemon running. Depends on the distribution, you should be able to check whether the service is running (e.g. systemctl status sshd).

Telnet also allows you to connect to a remote host but as opposed to SSH where the communication is encrypted, in telnet, the data is sent in clear text, so it doesn't considered to be secured because anyone on the network can see what exactly is sent, including passwords.

It means that the key of the remote host was changed and doesn't match the one that stored on the machine (in ~/.ssh/known_hosts).

• The ? matches any single character
• The * matches zero or more characters

1. An IP address
2. The word "error" or "failure"
3. Lines which end with a number

lines 1 and 3.

I consider this as a good blog post to read more about it: https://shapeshed.com/unix-exit-codes

Another way to ask this: what happens from the moment you turned on the server until you get a prompt

For each file (and directory) in Linux there is an inode, a data structure which stores meta data related to the file like its size, owner, permissions, etc.

File name (it's part of the directory file)

Run mount

cat /proc/mounts

Hard link is the same file, using the same inode. Soft link is a shortcut to another file, using a different inode.

False

True

True.

There are many answers for this question. One way is running df -T

du -sh

False. /tmp is cleared upon system boot while /var/tmp is cleared every a couple of days or not cleared at all (depends on distro).

One can use uptime or top

This article summarizes the load average topic in a great way

pidstat

iostat -xz 1

You can use the commands top and free

sar -n TCP,ETCP 1

ps -ef

You can achieve that by specifying & at the end of the command. As to why, since some commands/processes can take a lot of time to finish execution or run forever, you may want to run them in the background instead of waiting for them to finish before gaining control again in current session.

To view all available signals run kill -l

A background process. Most of these processes are waiting for requests or set of conditions to be met before actually running anything. Some examples: sshd, crond, rpcbind.

One reason it happens is when a parent process is programmed incorrectly. Every parent process should execute wait() to get the exit code from the child process which finished to run. But when the parent isn't checking for the child exit code, the child process can still exists although it finished to run.

You can also try closing/terminating the parent process. This will make the zombie process a child of init (1) which does periodic cleanups and will at some point clean up the zombie process.

If you mention at any point ps command with arugments, be familiar with what these arguments does exactly.

find /some_dir -iname *.yml -print0 | xargs -0 -r sed -i "s/1/2/g"

The ls executable is built for an incompatible architecture.

You can use the split command this way: split -l 25 some_file

This is a great article on the topic: https://www.computerhope.com/jargon/f/file-descriptor.htm

The loopback interface is a special, virtual network interface that your computer uses to communicate with itself. It is used mainly for diagnostics and troubleshooting, and to connect to servers running on the local machine.

False

Technically, yes.

Telnet is a type of client-server protocol that can be used to open a command line on a remote computer, typically a server. By default, all the data sent and received via telnet is transmitted in clear/plain text, therefore it should not be used as it does not encrypt any data between the client and the server.

One way would be ping6 ff02::1

• balance-rr: round robing bonding
• active-backup: a fault tolerance mode where only one is active
• balance-tlb: Adaptive transmit load balancing
• balance-alb: Adaptive load balancing

You can also run hostnamectl or hostname but that might print only a temporary hostname. The one in the file is the permanent one.

• dig
• host
• nslookup

ANSWER SECTION:
codingshell.com.	3515	IN	A	185.199.109.153

This is the TTL. When you lookup for an address using a domain/host name, your OS is performing DNS resolution by contacting DNS name servers to get the IP address of the host/domain you are looking for.
When you get a reply, this reply in cached in your OS for a certain period of time. This is period of time is also known as TTL and this is the meaning of 3515 number - it will be cached for 3515 seconds before removed from the cache and during that period of time, you'll get the value from the cache instead of asking DNS name servers for the address again.

In Fedora/CentOS/RHEL/Rocky it can be done with rpm or dnf commands. In Ubuntu it can be done with the apt command.

Package managers allow you to manage packages lifecycle as in installing, removing and updating the packages.
In addition, you can specify in a spec how a certain package will be installed - where to copy the files, which commands to run prior to the installation, post the installation, etc.

dnf provides /usr/bin/git

Systemd: systemctl enable [service_name] System V: update-rc.d [service_name] and add this line id:5678:respawn:/bin/sh /path/to/app to /etc/inittab Upstart: add Upstart init script at /etc/init/service.conf

1. SSH server is not installed
2. SSH server is not running

Nginx, Apache httpd.

Command to create users is useradd

Syntax: useradd [options] Username

There are 2 configuration files, which stores users information

1. /etc/passwd - Users information like, username, shell etc is stored in this file

2. /etc/shadow - Users password is stored in encrypted format

/etc/groups file stores the group name, group ID, usernames which are in secondary group.

passwd <username> is the command to set/change password of a user.

/etc/shadow file holds the passwords of the users in encryted format. NO, it is only visble to the root user

YES, we can create new user by manually adding an entry in the /etc/passwd file.

For example, if we need to create a user called john.

Step 1: Add an entry to /etc/passwd file, so user gets created.

echo "john:x:2001:2001::/home/john:/bin/bash" >> /etc/passwd

Step 2: Add an entry to /etc/group file, because every user belong to the primary group that has same name as the username.

echo "john:x:2001:" >> /etc/group

Step 3: Verify if the user got created

id john

/etc/passwd is a configuration file, which contains users information. Each entry in this file has, 7 fields,

username:password:UID:GID:Comment:home directory:shell

username - The name of the user.

password - This field is actually a placeholder of the password field. Due to security concerns, this field does not contain the password, just a placeholder (x) to the encrypted password stored in /etc/shadow file.

UID - User ID of the user.

GID - Group ID

Comment - This field is to provide description about the user.

home directory - Abousulte path of the user's home directory. This directory gets created once the user is added.

shell - This field contains the absolute path of the shell that will be used by the respective user.

adduser user_name --shell=/bin/false --no-create-home You can also add a user and then edit /etc/passwd.

su command. Use su - to switch to root

UID of root user is 0

Default values of UID_MIN and UID_MAX in /etc/login.defs UID_MIN is 1000 UID_MAX is 60000

Actually, we can change this value. But UID < 1000 are reserved for system accounts. Therefore, as per the default configuration, for regular user UID starts from 1000.

Re-install the OS IS NOT the right answer :)

/etc/skel is a directory, that contains files or directories, so when a new user is created, these files/directories created under /etc/skel will be copied to user's home directory.

Using the last command.

useradd - Command for creating new users usermod - Modify the users setting whoami - Outputs, the username that we are currently logged in id - Prints the

The user you are using isn't defined locally but originates from services like LDAP.
You can verify with: getent passwd

You can also use nproc for number of processors

dmidecoode

lsblk

True. Only in kernel space they have full access to hardware resources.

openssl genrsa -aes256 -out ca-private-key.pem 4096

openssl req -new -x509 -days 730 -key ca-private-key.pem -sha256 -out ca.pem

• Process ID namespaces: these namespaces include independent set of process IDs
• Mount namespaces: Isolation and control of mountpoints
• Network namespaces: Isolates system networking resources such as routing table, interfaces, ARP table, etc.
• UTS namespaces: Isolate host and domains
• IPC namespaces: Isolates interprocess communications
• User namespaces: Isolate user and group IDs
• Time namespaces: Isolates time machine

True. Inside the namespace it's PID 1 while to the parent namespace the PID is a different one.

False. The opposite is true. Parent PID namespace is aware and has visibility of processes in child PID namespace and child PID namespace has no visibility as to what is going on in the parent PID namespace.

False. Network namespace has its own interfaces and routing table. There is no way (without creating a bridge for example) for one network namespace to reach another.

True

False. In every child user namespace, it's possible to have a separate root user with uid of 0.

In time namespaces processes can use different system time.

• KVM
• XEN
• VirtualBox
• Linux-VServer
• User-mode Linux
• ...

• Red Hat Article - What is KVM?

From the official docs Hypervisor supported docs

From Wikipedia: "AWK is domain-specific language designed for text processing and typically used as a data extraction and reporting tool"

awk '{print $4}' file

awk 'length($0) > 79' file

Using system calls

• A program executes a trap instruction. The instruction jump into the kernel while raising the privileged level to kernel space.
• Once in kernel space, it can perform any privileged operation
• Once it's finished, it calls a "return-from-trap" instruction which returns to user space while reducing back the privilege level to user space.

fork() is used for creating a new process. It does so by cloning the calling process but the child process has its own PID and any memory locks, I/O operations and semaphores are not inherited.

• On success, the PID of the child process in parent and 0 in child process
• On error, -1 in the parent

Not enough memory to create a new process

wait() is used by a parent process to wait for the child process to finish execution. If wait is not used by a parent process then a child process might become a zombie process.

The kernel notifies the parent by sending the SIGCHLD to the parent.

The waitpid() is a non-blocking version of the wait() function.
It also supports using library routine (e.g. system()) to wait a child process without messing up with other children processes for which the process has not waited.

True in most cases though there are cases where wait() returns before the child exits.

It transforms the current running program into another program.
Given the name of an executable and some arguments, it loads the code and static data from the specified executable and overwrites its current code segment and current static code data. After initializing its memory space (like stack and heap) the OS runs the program passing any arguments as the argv of that process.

True
Since a succesful exec replace the current process, it can't return anything to the process that made the call.

fork(), exec() and the wait() system call is also included in this workflow.

Executes a program. The program is passed as a filename (or path) and must be a binary executable or a script.

"Pipes provide a unidirectional interprocess communication channel. A pipe has a read end and a write end. Data written to the write end of a pipe can be read from the read end of the pipe. A pipe is created using pipe(2), which returns two file descriptors, one referring to the read end of the pipe, the other referring to the write end."

• getline() originates in GNU C library and used to read lines from input stream and stores those lines in the buffer

This way provides a lot of flexibility. It allows the shell for example, to run code after the call to fork() but before the call to exec(). Such code can be used to alter the environment of the program it about to run.

The shell figures out, using the PATH variable, where the executable of the command resides in the filesystem. It then calls fork() to create a new child process for running the command. Once the fork was executed successfully, it calls a variant of exec() to execute the command and finally, waits the command to finish using wait(). When the child completes, the shell returns from wait() and prints out the prompt again.

• dd if=/dev/urandom of=new_file.txt bs=2MB count=1
• truncate -s 2M new_file.txt
• fallocate -l 2097152 new_file.txt

open("/my/file") = 5
read(5, "file content")

These system calls are reading the file /my/file and 5 is the file descriptor number.

From wikipedia: a context switch is the process of storing the state of a process or thread, so that it can be restored and resume execution at a later point

Another common way to task this questions is "what part of the tcp header does traceroute modify?"

This is a good article about the topic: https://ops.tips/blog/how-linux-creates-sockets

MemFree - The amount of unused physical RAM in your system MemAvailable - The amount of available memory for new workloads (without pushing system to use swap) based on MemFree, Active(file), Inactive(file), and SReclaimable.

• Kernel
• Utilities
• Services
• Software/Packages Management

echo $line | sed 's/.*\[//g;s/].*//g;s/:.*//g'

One way is to run the following: cat /proc/sys/kernel/random/uuid

• A collection of packages - kernel, GNU, third party apps, ...
• Sometimes distributions store some information on the distribution in /etc/*-release file
  • For example for Red Hat distribution it will be /etc/redhat-release and for Amazon it will be /etc/os-release
  • lsb_release is a common command you can use in multiple different distributions

ls, wc, dd, df, du, ps, ip, cp, cd ...

• touch new_file
• echo "" > new_file

$OLDPWD

• ls
• find .
• echo *

A good answer can be found here

X=2 for example. But this will persist to new shells. To have it in new shells as well, use export X=2

It's used in commands to mark the end of commands options. One common example is when used with git to discard local changes: git checkout -- some_file

/dev

GPL v2

"responsible for making it easy to run programs (even allowing you to seemingly run many at the same time), allowing programs to share memory, enabling programs to interact with devices, and other fun stuff like that".

A process is a running program. A program is one or more instructions and the program (or process) is executed by the operating system.

• Create - allow to create new processes
• Delete - allow to remove/destroy processes
• State - allow to check the state of the process, whether it's running, stopped, waiting, etc.
• Stop - allow to stop a running process

• The OS is reading program's code and any additional relevant data
• Program's code is loaded into the memory or more specifically, into the address space of the process.
• Memory is allocated for program's stack (aka run-time stack). The stack also initialized by the OS with data like argv, argc and parameters to main()
• Memory is allocated for program's heap which is required for dynamically allocated data like the data structures linked lists and hash tables
• I/O initialization tasks are performed, like in Unix/Linux based systems where each process has 3 file descriptors (input, output and error)
• OS is running the program, starting from main()

False. It was true in the past but today's operating systems perform lazy loading which means only the relevant pieces required for the process to run are loaded first.

• Running - it's executing instructions
• Ready - it's ready to run but for different reasons it's on hold
• Blocked - it's waiting for some operation to complete. For example I/O disk request

• I/O operations (e.g. Reading from a disk)
• Waiting for a packet from a network

Even when using a system with one physical CPU, it's possible to allow multiple users to work on it and run programs. This is possible with time sharing where computing resources are shared in a way it seems to the user the system has multiple CPUs but in fact it's simply one CPU shared by applying multiprogramming and multi-tasking.

Somewhat the opposite of time sharing. While in time sharing a resource is used for a while by one entity and then the same resource can be used by another resource, in space sharing the space is shared by multiple entities but in a way where it's not being transferred between them.
It's used by one entity until this entity decides to get rid of it. Take for example storage. In storage, a file is yours until you decide to delete it.

CPU scheduler

• Allocating memory
• Schedule processes
• Control CPU

True

Buffer: Reserved place in RAM which is used to hold data for temporary purposes Cache: Cache is usually used when processes reading and writing to the disk to make the process faster by making similar data used by different programs easily accessible.

Virtualization uses software to create an abstraction layer over computer hardware that allows the hardware elements of a single computer—processors, memory, storage and more - to be divided into multiple virtual computers, commonly called virtual machines (VMs).

Read more here

Hosted hypervisors and bare-metal hypervisors.

On the other hand, there will probably be some limitation regarding loading (any) drivers so a hosted hypervisor will usually benefit from having a better hardware compatibility.

Operating system virtualization Network functions virtualization Desktop virtualization

Yes, it's a operating-system-level virtualization, where the kernel is shared and allows to use multiple isolated user-spaces instances.

The introduction of virtual machines allowed companies to deploy multiple business applications on the same hardware while each application is separated from each other in secured way, where each is running on its own separate operating system.

• False
• True
• False
• TypeError

bool("") -> evaluates to False
bool(" ") -> evaluates to True

It evaluates to True.
The reason is that the two created empty list are different objects. x is y only evaluates to true when x and y are the same object.

0

True

string.isdigit

string.rstrip('/')

• abcabcabc
• TypeError
• TypeError
• "abc"
• ""

def my_function():
    return 5

You can then assign a function to a variables like this x = my_function or you can return functions as return values like this return my_function

1. Translation lookup in i18n
2. Hold the result of the last executed expression or statement in the interactive interpreter.
3. As a general purpose "throwaway" variable name. For example: x, y, _ = get_data() (x and y are used but since we don't care about third variable, we "threw it away").

A tuple is a built-in data type in Python. It's used for storing multiple items in a single variable.

List, as opposed to a tuple, is a mutable data type. It means we can modify it and at items to it.

x.append(2)

len(sone_list)

some_list[-1]

Don't use append unless you would like the list as one item.

my_list[0:3] = []

• sorted(list) will return a new list (original list doesn't change)

• list.sort() will return None but the list is change in-place

• sorted() works on any iterable (Dictionaries, Strings, ...)

• list.sort() is faster than sorted(list) in case of Lists

There are many ways of solving this problem:
# Note: :list and -> bool are just python typings, they are not needed for the correct execution of the algorithm.

Taking advantage of sets and len:

def is_unique(l:list) -> bool:
    return len(set(l)) == len(l)

This one is can be seen used in other programming languages.

def is_unique2(l:list) -> bool:
    seen = []

    for i in l:
        if i in seen:
            return False
        seen.append(i)
    return True

Here we just count and make sure every element is just repeated once.

def is_unique3(l:list) -> bool:
    for i in l:
        if l.count(i) > 1:
            return False
    return True

This one might look more convulated but hey, one liners.

def is_unique4(l:list) -> bool:
    return all(map(lambda x: l.count(x) < 2, l))

my_dict = dict(x=1, y=2) OR my_dict = {'x': 1, 'y': 2} OR my_dict = dict([('x', 1), ('y', 2)])

del my_dict['some_key'] you can also use my_dict.pop('some_key') which returns the value of the key.

Solution

Using the re module

It will reverse the string, so x would be equal to azzip.

mario

for i in range(3, 3):
   print(i)

No output :)

the_list.sort(key=lambda x: x[1])

pdb :D

Short answer is: It returns a None object.

We could go a bit deeper and explain the difference between

def a ():
    return

>>> None

And

def a ():
    pass

>>> None

Or we could be asked this as a following question, since they both give the same result.

We could use the dis module to see what's going on:

  2           0 LOAD_CONST               0 (<code object a at 0x0000029C4D3C2DB0, file "<dis>", line 2>)
              2 LOAD_CONST               1 ('a')
              4 MAKE_FUNCTION            0
              6 STORE_NAME               0 (a)

  5           8 LOAD_CONST               2 (<code object b at 0x0000029C4D3C2ED0, file "<dis>", line 5>)
             10 LOAD_CONST               3 ('b')
             12 MAKE_FUNCTION            0
             14 STORE_NAME               1 (b)
             16 LOAD_CONST               4 (None)
             18 RETURN_VALUE

Disassembly of <code object a at 0x0000029C4D3C2DB0, file "<dis>", line 2>:
  3           0 LOAD_CONST               0 (None)
              2 RETURN_VALUE

Disassembly of <code object b at 0x0000029C4D3C2ED0, file "<dis>", line 5>:
  6           0 LOAD_CONST               0 (None)
              2 RETURN_VALUE

An empty return is exactly the same as return None and functions without any explicit return will always return None regardless of the operations, therefore

def sum(a, b):
    global c
    c = a + b

>>> None

def print_list(self): node = self.head while(node): print(node.data) node = node.next

Read about descriptors here

This is a good reference https://docs.python.org/3/library/datatypes.html

def wee(word):
    return word

def oh(f):
    return f + "Ohh"

>>> oh(wee("Wee"))
<<< Wee Ohh

every time, that's what decorators do, they introduce syntax to write all of this on the go, using the keyword '@'.

Google: "Monitoring is one of the primary means by which service owners keep track of a system’s health and availability".

This approach require from a human to always check why the value exceeded and how to handle it while today, it is more effective to notify people only when they need to take an actual action. If the issue doesn't require any human intervention, then the problem can be fixed by some processes running in the relevant environment.

Alerts
Tickets
Logging

From Prometheus documentation: "if you need 100% accuracy, such as for per-request billing".

Prometheus server is responsible for scraping and storing the data
Push gateway is used for short-lived jobs
Alert manager is responsible for alerts ;)

Go also has good community.

With var x int = 2 we are setting the variable type to integer while with x := 2 we are letting Go figure out by itself the type.

False. We can't redeclare variables but yes, we must used declared variables.

• fmt - formatted I/O

package main

import "fmt"

func main() {
    var x int = 101
    var y string
    y = string(x)
    fmt.Println(y)
}

It looks what unicode value is set at 101 and uses it for converting the integer to a string. If you want to get "101" you should use the package "strconv" and replace y = string(x) with y = strconv.Itoa(x)

package main

func main() {
    var x = 2
    var y = 3
    const someConst = x + y
}

Constants in Go can only be declared using constant expressions. But x, y and their sum is variable.
const initializer x + y is not a constant

package main

import "fmt"

const (
	x = iota
	y = iota
)
const z = iota

func main() {
	fmt.Printf("%v\n", x)
	fmt.Printf("%v\n", y)
	fmt.Printf("%v\n", z)
}

Go's iota identifier is used in const declarations to simplify definitions of incrementing numbers. Because it can be used in expressions, it provides a generality beyond that of simple enumerations.
x and y in the first iota group, z in the second.
Iota page in Go Wiki

It avoids having to declare all the variables for the returns values. It is called the blank identifier.
answer in SO

package main

import "fmt"

const (
	_ = iota + 3
	x
)

func main() {
	fmt.Printf("%v\n", x)
}

Since the first iota is declared with the value 3 ( + 3), the next one has the value 4

package main

import (
	"fmt"
	"sync"
	"time"
)

func main() {
	var wg sync.WaitGroup

	wg.Add(1)
	go func() {
		time.Sleep(time.Second * 2)
		fmt.Println("1")
		wg.Done()
	}()

	go func() {
		fmt.Println("2")
	}()

	wg.Wait()
	fmt.Println("3")
}

Golang package sync

package main

import (
	"fmt"
)

func mod1(a []int) {
	for i := range a {
		a[i] = 5
	}

	fmt.Println("1:", a)
}

func mod2(a []int) {
	a = append(a, 125) // !

	for i := range a {
		a[i] = 5
	}

	fmt.Println("2:", a)
}

func main() {
	s1 := []int{1, 2, 3, 4}
	mod1(s1)
	fmt.Println("1:", s1)

	s2 := []int{1, 2, 3, 4}
	mod2(s2)
	fmt.Println("2:", s2)
}

Aritcle about arrays, Blog post about append

package main

import (
	"container/heap"
	"fmt"
)

// An IntHeap is a min-heap of ints.
type IntHeap []int

func (h IntHeap) Len() int           { return len(h) }
func (h IntHeap) Less(i, j int) bool { return h[i] < h[j] }
func (h IntHeap) Swap(i, j int)      { h[i], h[j] = h[j], h[i] }

func (h *IntHeap) Push(x interface{}) {
	// Push and Pop use pointer receivers because they modify the slice's length,
	// not just its contents.
	*h = append(*h, x.(int))
}

func (h *IntHeap) Pop() interface{} {
	old := *h
	n := len(old)
	x := old[n-1]
	*h = old[0 : n-1]
	return x
}

func main() {
	h := &IntHeap{4, 8, 3, 6}
	heap.Init(h)
	heap.Push(h, 7)

  fmt.Println((*h)[0])
}

Golang container/heap package

The main difference is that SQL databases are structured (data is stored in the form of tables with rows and columns - like an excel spreadsheet table) while NoSQL is unstructured, and the data storage can vary depending on how the NoSQL DB is set up, such as key-value pair, document-oriented, etc.

• Heterogeneous data which changes often
• Data consistency and integrity is not top priority
• Best if the database needs to scale rapidly

• mongoexport
• programming languages

SQL (Structured Query Language) is a standard language for relational databases (like MySQL, MariaDB, ...).
It's used for reading, updating, removing and creating data in a relational database.

The main difference is that SQL databases are structured (data is stored in the form of tables with rows and columns - like an excel spreadsheet table) while NoSQL is unstructured, and the data storage can vary depending on how the NoSQL DB is set up, such as key-value pair, document-oriented, etc.

Additionally, since NoSQL does not adhere to the strict table with columns and rows structure that Relational Databases require, you can store different data types together.

Select *
From Customers;

Select Items_in_cart
From Customers
Where Customer_Name = "John Smith";

Select SUM(Cash_spent_to_Date) as SUM_CASH
From Customers;

Select count(1) as Number_of_People_w_items
From Customers
where Items_in_cart > 0;

You would join them on the unique key. In this case, the unique key is Customer_ID in both the Customers table and Orders table

The Sum of all the purchases of cat food weren't readily available, so we used a with statement to create the pseudo table to retrieve the sum of the prices spent by each customer, then join the table normally.

SELECT count(*)                             SELECT count(*)
FROM shawarma_purchases                     FROM shawarma_purchases
WHERE                               vs.     WHERE
  YEAR(purchased_at) == '2017'              purchased_at >= '2017-01-01' AND
                                            purchased_at <= '2017-31-12'

SELECT count(*)
FROM shawarma_purchases
WHERE
  purchased_at >= '2017-01-01' AND
  purchased_at <= '2017-31-12'

When you use a function (YEAR(purchased_at)) it has to scan the whole database as opposed to using indexes and basically the column as it is, in its natural state.

Microsoft Docs: "The Azure portal is a web-based, unified console that provides an alternative to command-line tools. With the Azure portal, you can manage your Azure subscription by using a graphical user interface."

Microsoft Docs: "Azure marketplace helps connect users with Microsoft partners, independent software vendors, and startups that are offering their solutions and services, which are optimized to run on Azure."

An availability set is a logical grouping of VMs that allows Azure to understand how your application is built to provide redundancy and availability. It is recommended that two or more VMs are created within an availability set to provide for a highly available application and to meet the 99.95% Azure SLA.

• Azure Virtual Machines
• Azure Batch
• Azure Service Fabric
• Azure Container Instances
• Azure Virtual Machine Scale Set?s

Windows or Linux virtual machines

Scaling Linux or Windows virtual machines used in Azure

Azure Functions is the serverless compute service of Azure.

Running containerized applications (without the need to provision virtual machines).

Running parallel and high-performance computing applications

• Provides security recommendations based on your usage
• Monitors security settings and continuously all the services
• Analyzes and identifies potential inbound attacks
• Detects and blocks malware using machine learning

Azure AD is a cloud-based identity service. You can use it as a standalone service or integrate it with existing Active Directory service you already running.

startap-script

• It is the managed kubernetes service on GCP for deploying, managing and scaling containerised applications using Google infrastructure.

fun fact: Anthos is flower in greek, they grow in the ground (earth) but need rain from the clouds to flourish.

• Infrastructure management - Google Kubernetes Engine (GKE)
• Cluster management - GKE, Ingress for Anthos
• Service management - Anthos Service Mesh
• Policy enforcement - Anthos Config Management, Anthos Enterprise Data Protection, Policy Controller
• Application deployment - CI/CD tools like Cloud Build, GitLab
• Application development - Cloud Code

• Google Kubernetes Engine (GKE)

On GCP the kubernetes api-server is the only control plane component exposed to customers whilst compute engine manages instances in the project.

• Networking load balancing for L4 and HTTP(S) Load Balancing for L7 which are both managed services that do not require additional configuration.
• Ingress for Anthos which allows the ability to deploy a load balancer that serves an application across multiple clusters on GKE

• Yes, Anthos on AWS is now GA. For more read here

• Control plane security - GCP manages and maintains the K8s control plane out of the box. The user can secure the api-server by using master authorized networks and private clusters. These allow the user to disable access on the public IP address by assigning a private IP address to the master.
• Node security - By default workloads are provisioned on Compute engine instances that use Google's Container Optimised OS. This operating system implements a locked-down firewall, limited user accounts with root disabled and a read-only filesystem. There is a further option to enable GKE Sandbox for stronger isolation in multi-tenant deployment scenarios.
• Network security - Within a created cluster VPC, Anthos GKE leverages a powerful software-defined network that enables simple Pod-to-Pod communications. Network policies allow locking down ingress and egress connections in a given namespace. Filtering can also be implemented to incoming load-balanced traffic for services that require external access, by supplying whitelisted CIDR IP ranges.
• Workload security - Running workloads run with limited privileges, default Docker AppArmor security policies are applied to all Kubernetes Pods. Workload identity for Anthos GKE aligns with the open source kubernetes service accounts with GCP service account permissions.
• Audit logging - Adminstrators are given a way to retain, query, process and alert on events of the deployed environments.

• Google Cloud Virtual Private Network (Cloud VPN) - this is for secure networking
• Google Cloud Key Management Service (Cloud KMS) - for key management

• This is when pods can directly talk to each other within a cluster, but cannot be reached from outside the cluster thus forming an "island" within the network that is not connected to the external network.

1. An importer that reads from a central Git repository
2. A component that synchronises stored configuration data into K8s objects
3. A component that monitors drift between desired and actual cluster configurations with a capability of reconciliation when need rises.

It follows common modern software development practices which makes cluster configuration, management and policy changes auditable, revertable, and versionable easily enforcing IT governance and unifying resource management in an organisation.

• It is a suite of tools that assist in monitoring and managing deployed services on Anthos of all shapes and sizes whether running in cloud, hybrid or multi-cloud environments. It leverages the APIs and core components from Istio, a highly configurable and open-source service mesh platform.

1. Data plane - it consists of a set of distributed proxies that mediate all inbound and outbound network traffic between individual services which are configured using a centralised control plane and an open API
2. Control plane - is a fully managed offering outside of Anthos GKE clusters to simplify management overhead and ensure highest possible availability.

1. Traffic Director - it is GCP's fully managed service mesh traffic control plane, responsible for translating Istio API objects into configuration information for the distributed proxies, as well as directing service mesh ingress and egress traffic
2. Managed CA - is a centralised certificate authority responsible for providing SSL certificates to each of the distributed proxies, authentication information and distributing secrets
3. Operations tooling - formerly stackdriver, provides a managed ingestion point for observability and telemetry, specifically monitoring, tracing and logging data generated by each of the proxies. This powers the observability dashboard for operators to visually inspect their services and service dependencies assisting in the implementation of SRE best practices for monitoring SLIs and establishing SLOs.

• Uniform observability - the data plane reports service to service communication back to the control plane generating a service dependency graph. Traffic inspection by the proxy inserts headers to facilitate distributed tracing, capturing and reporting service logs together with service-level metrics (i.e latency, errors, availability).
• Operational agility - fine-grained controls for managing the flow of inter-mesh (north-south) and intra-mesh (east-west) traffic are provided.
• Policy-driven security - policies can be enforced consistently across diverse protocols and runtimes as service communications are secured by default.

• Traffic splitting across differing service versions for canary or A/B testing
• Circuit breaking to prevent cascading failures
• Fault injection to help build resilient and fault-tolerant deployments
• HTTP header-based traffic steering between individual services or versions

It is part of the Anthos stack that brings a serverless container experience to Anthos, offering a high-level platform experience on top of K8s clusters. It is built with Knative, an open-source operator for K8s that brings serverless application serving and eventing capabilities.

• Easy migration from K8s deployments - Without Cloud Run, platform engineers have to configure deployment, service, and HorizontalPodAutoscalers(HPA) objects to a loadbalancer and autoscaling. If application is already serving traffic it becomes hard to change configurations or roll back efficiently. Using Cloud Run all this is managed thus the Knative service manifest describes the application to be autoscaled and loadbalanced
• Autoscaling - a sudden traffic spike may cause application containers in K8s to crash due to overload thus an efficient automated autoscaling is executed to serve the high volume of traffic
• Networking - it has built-in load balancing capabilities and policies for traffic splitting between multiple versions of an application.
• Releases and rollouts - supports the notion of the Knatibe API's revisions which describe new versions or different configurations of your application and canary deployments by splitting traffic.
• Monitoring - observing and recording metrics such as latency, error rate and requests per second.

• Rapid, request-based autoscaling - default autoscalers monitor request metrics which allows Cloud Run for Anthos to handle spiky traffic patterns smoothly
• Concurrency controls - limits such as max in-flight requests per container are enforced to ensure the container does not become overloaded and crash. More containers are added to handle the spiky traffic, buffering the requests.
• Scale to zero - if an application is inactive for a while Cloud Run scales it down to zero to reduce its footprint. Alternatively one can turn off scale-to-zero to prevent cold starts.

• Machine learning model predictions e.g Tensorflow serving containers
• API gateways, API middleware, web front ends and Microservices
• Event handlers, ETL

• Nova - Manage virtual instances
• Cinder - Block Storage
• Keystone - Authentication service across the cloud

• Glance - Images Service. Also used for copying or snapshot instances
• Horizon - GUI for viewing and modifying resources
• Cinder - Block Storage
• Nova - Manage virtual instances

You can read about TripleO right here

• Used to provision and manage virtual instances
• It supports Multi-Tenancy in different levels - logging, end-user control, auditing, etc.
• Highly scalable
• Authentication can be done using internal system or LDAP
• Supports multiple types of block storage
• Tries to be hardware and hypervisor agnostice

• nova-api - the server which serves metadata and compute APIs
• the different Nova components communicate by using a queue (Rabbitmq usually) and a database
• a request for creating an instance is inspected by nova-scheduler which determines where the instance will be created and running
• nova-compute is the component responsible for communicating with the hypervisor for creating the instance and manage its lifecycle

• One of the core component of OpenStack and a standalone project
• Neutron focused on delivering networking as a service
• With Neutron, users can set up networks in the cloud and configure and manage a variety of network services
• Neutron interacts with:
  • Keystone - authorize API calls
  • Nova - nova communicates with neutron to plug NICs into a network
  • Horizon - supports networking entities in the dashboard and also provides topology view which includes networking details

• neutron-l3-agent - L3/NAT forwarding (provides external network access for VMs for example)
• neutron-dhcp-agent - DHCP services
• neutron-metering-agent - L3 traffic metering
• neutron-*-agtent - manages local vSwitch configuration on each compute (based on chosen plugin)
• neutron-server - exposes networking API and passes requests to other plugins if required

• Management Network - used for internal communication between OpenStack components. Any IP address in this network is accessible only within the datacetner
• Guest Network - used for communication between instances/VMs
• API Network - used for services API communication. Any IP address in this network is publicly accessible
• External Network - used for public communication. Any IP address in this network is accessible by anyone on the internet

There are many reasons for that. One for example: you can't remove router if there are active ports assigned to it.

• Glance is the OpenStack image service
• It handles requests related to instances disks and images
• Glance also used for creating snapshots for quick instances backups
• Users can use Glance to create new images or upload existing ones

• glance-api - responsible for handling image API calls such as retrieval and storage. It consists of two APIs: 1. registry-api - responsible for internal requests 2. user API - can be accessed publicly
• glance-registry - responsible for handling image metadata requests (e.g. size, type, etc). This component is private which means it's not available publicly
• metadata definition service - API for custom metadata
• database - for storing images metadata
• image repository - for storing images. This can be a filesystem, swift object storage, HTTP, etc.

• Swift is Object Store service and is an highly available, distributed and consistent store designed for storing a lot of data
• Swift is distributing data across multiple servers while writing it to multiple disks
• One can choose to add additional servers to scale the cluster. All while swift maintaining integrity of the information and data replications.

Not by default. Object Storage API limits the maximum to 5GB per object but it can be adjusted.

• Container - Defines a namespace for objects.
• Account - Defines a namespace for containers
• Object - Data content (e.g. image, document, ...)

False. Two objects can have the same name if they are in different containers.

• Cinder is OpenStack Block Storage service
• It basically provides used with storage resources they can consume with other services such as Nova
• One of the most used implementations of storage supported by Cinder is LVM
• From user perspective this is transparent which means the user doesn't know where, behind the scenes, the storage is located or what type of storage is used

• cinder-api - receives API requests
• cinder-volume - manages attached block devices
• cinder-scheduler - responsible for storing volumes

• Role - A list of rights and privileges determining what a user or a project can perform
• Tenant/Project - Logical representation of a group of resources isolated from other groups of resources. It can be an account, organization, ...
• Service - An endpoint which the user can use for accessing different resources
• Endpoint - a network address which can be used to access a certain OpenStack service
• Token - Used for access resources while describing which resources can be accessed by using a scope

• Name
• ID number
• Type
• Description

• PublicURL - Publicly accessible through public internet
• InternalURL - Used for communication between services
• AdminURL - Used for administrative management

A list of services and their endpoints

• Swift - highly available, distributed, eventually consistent object/blob store
• Sahara - Manage Hadoop Clusters
• Ironic - Bare Metal Provisioning
• Trove - Database as a service that runs on OpenStack
• Aodh - Alarms Service
• Ceilometer - Track and monitor usage

• Database as a service which runs on OpenStack - Trove
• Bare Metal Provisioning - Ironic
• Track and monitor usage - Ceilometer
• Alarms Service - Aodh
• Manage Hadoop Clusters
• Manage Hadoop Clusters - Sahara
• highly available, distributed, eventually consistent object/blob store - Swift

• You can't have OpenStack deployed without Keystone
• It Provides identity, policy and token services
  • The authentication provided is for both users and services
  • The authorization supported is token-based and user-based.
• There is a policy defined based on RBAC stored in a JSON file and each line in that file defines the level of access to apply

• There is a service API and admin API through which Keystone gets requests
• Keystone has four backends:
  • Token Backend - Temporary Tokens for users and services
  • Policy Backend - Rules management and authorization
  • Identity Backend - users and groups (either standalone DB, LDAP, ...)
  • Catalog Backend - Endpoints
• It has pluggable environment where you can integrate with:
  • LDAP
  • KVS (Key Value Store)
  • SQL
  • PAM
  • Memcached

• Keystone gets a call/request and checks whether it's from an authorized user, using username, password and authURL
• Once confirmed, Keystone provides a token.
• A token contains a list of user's projects so there is no to authenticate every time and a token can submitted instead

• nova-api - responsible for managing requests/calls
• nova-compute - responsible for managing instance lifecycle
• nova-conductor - Mediates between nova-compute and the database so nova-compute doesn't access it directly

• Nova-novncproxy - Access through VNC connections
• Nova-spicehtml5proxy - Access through SPICE
• Nova-xvpvncproxy - Access through a VNC connection

• Django-based project focusing on providing an OpenStack dashboard and the ability to create additional customized dashboards
• You can use it to access the different OpenStack services resources - instances, images, networks, ...
  • By accessing the dashboard, users can use it to list, create, remove and modify the different resources
• It's also highly customizable and you can modify or add to it based on your needs

• API is backward compatible
• There are three type of dashboards: user, system and settings
• It provides core support for all OpenStack core projects such as Neutron, Nova, etc. (out of the box, no need to install extra packages or plugins)
• Anyone can extend the dashboards and add new components
• Horizon provides templates and core classes from which one can build its own dashboard

Elasticserach, Logstash and Kibana are also known as the ELK stack.

"Elasticsearch is a distributed document store. Instead of storing information as rows of columnar data, Elasticsearch stores complex data structures that have been serialized as JSON documents"

"Logstash is a powerful, flexible pipeline that collects, enriches and transports data. It works as an extract, transform & load (ETL) tool for collecting log messages."

Beats are lightweight data shippers. These data shippers installed on the client where the data resides. Examples of beats: Filebeat, Metricbeat, Auditbeat. There are much more.

"Kibana is an open source analytics and visualization platform designed to work with Elasticsearch. You use Kibana to search, view, and interact with data stored in Elasticsearch indices. You can easily perform advanced data analysis and visualize your data in a variety of charts, tables, and maps."

1. The data logged by the application is picked by filebeat and sent to logstash
2. Logstash process the log based on the defined filters. Once done, the output is sent to Elasticsearch
3. Elasticsearch stores the document it got and the document is indexed for quick future access
4. The user creates visualizations in Kibana which based on the indexed data
5. The user creates a dashboard which composed out of the visualization created in the previous step

This is where data is stored and also where different processing takes place (e.g. when you search for a data).

While there can be multiple master nodes in reality only of them is the elected master node.

A node which responsible for parsing the data. In case you don't use logstash then this node can recieve data from beats and parse it, similarly to how it can be parsed in Logstash.

A Coordinating node responsible for routing requests out and in to the cluser (data nodes).

• Data is stored in an index
• The index is spread across the cluster using shards

"An index can be thought of as an optimized collection of documents and each document is a collection of fields, which are the key-value pairs that contain your data"

An index is split into shards and documents are hashed to a particular shard. Each shard may be on a different node in a cluster and each one of the shards is a self contained index.
This allows Elasticsearch to scale to an entire cluster of servers.

"An inverted index lists every unique word that appears in any document and identifies all of the documents each word occurs in."

Continuing with the comparison to SQL/NoSQL a Document in Elastic is a row in table in the case of SQL or a document in a collection in the case of NoSQL. As in NoSQL a Document is a JSON object which holds data on a unit in your app. What is this unit depends on the your app. If your app related to book then each document describes a book. If you are app is about shirts then each document is a shirt.

Red means some data is unavailable. Yellow can be caused by running single node cluster instead of multi-node.

"Each indexed field has a dedicated, optimized data structure. For example, text fields are stored in inverted indices, and numeric and geo fields are stored in BKD trees."

• _index
• _id
• _type

• You can optimize fields for partial matching
• You can define custom formats of known fields (e.g. date)
• You can perform language-specific analysis

In a network/cloud environment where failures can be expected any time, it is very useful and highly recommended to have a failover mechanism in case a shard/node somehow goes offline or disappears for whatever reason. To this end, Elasticsearch allows you to make one or more copies of your index’s shards into what are called replica shards, or replicas for short.

Term Frequency is how often a term appears in a given document and Document Frequency is how often a term appears in all documents. They both are used for determining the relevance of a term by calculating Term Frequency / Document Frequency.

"The index is actively being written to". More about the phases here

It creates customer index if it doesn't exists and adds a new document with the field name which is set to "John Dow". Also, if it's the first document it will get the ID 1.

1. If name value was different then it would update "name" to the new value
2. In any case, it bumps version field by one

Bulk API is used when you need to index multiple documents. For high number of documents it would be significantly faster to use rather than individual requests since there are less network roundtrips.

"In a filter context, a query clause answers the question “Does this document match this query clause?” The answer is a simple Yes or No — no scores are calculated. Filter context is mostly used for filtering structured data"

A small-scale architecture of elastic will consist of the elastic stack as it is. This means we will have beats, logstash, elastcsearch and kibana.
A production environment with large amounts of data can include some kind of buffering component (e.g. Reddis or RabbitMQ) and also security component such as Nginx.

• Input Plugins - how to collect data from different sources
• Filter Plugins - processing data
• Output Plugins - push data to different outputs/services/platforms

A logstash plugin which modifies information in one format and immerse it in another.

The raw data as it is stored in the index. You can search and filter it.

Total number of documents matching the search results. If not query used then simply the total number of documents.

"Visualize" is where you can create visual representations for your data (pie charts, graphs, ...)

Read here

False. One harvester harvests one file.

You can generate certificates with the provided elastic utils and change configuration to enable security using certificates model.

Another definition: "Systems that are physically separated, but logically connected"

• Network
• CPU
• Memory
• Disk

• Availability: Every request receives a response (it doesn't has to be the most recent data)
• Consistency: Every request receives a response with the latest/most recent data
• Partition tolerance: Even if some the data is lost/dropped, the system keeps running

What are the problems with the following design? How to improve it?

• Add another load balancer
• Use DNS A record for both load balancers
• Use message queue

It's an architecture in which data is and retrieved from a single, non-shared, source usually exclusively connected to one node as opposed to architectures where the request can get to one of many nodes and the data will be retrieved from one shared location (storage, memory, ...).

TODO: add more details!

"An explicitly and purposefully defined interface designed to be invoked over a network that enables software developers to get programmatic access to data and functionality within an organization in a controlled and comfortable way."

"An API specification provides a broad understanding of how an API behaves and how the API links with other APIs. It explains how the API functions and the results to expect when using the API"

"An API definition is similar to an API specification in that it provides an understanding of how an API is organized and how the API functions. But the API definition is aimed at machine consumption instead of human consumption of APIs."

Automation is the act of automating tasks to reduce human intervention or interaction in regards to IT technology and systems.
While automation focuses on a task level, Orchestration is the process of automating processes and/or workflows which consists of multiple tasks that usually across multiple systems.

• Authorization
• Logging
• Authentication
• Ordering
• Front-end
• Back-end ...

Data about data. Basically, it describes the type of information that an underlying data will hold.

I can't answer this for you :)

Domain Specific Language (DSLs) are used to create a customised language that represents the domain such that domain experts can easily interpret it.

Data serialization language used by many technologies today like Kubernetes, Ansible, etc.

True. Because YAML is superset of JSON.

someMultiLineString: |
  look mama
  I can write a multi-line string
  I love YAML

It's good for use cases like writing a shell script where each line of the script is a different command.

Wikipedia: "In computing, firmware is a specific class of computer software that provides the low-level control for a device's specific hardware. Firmware, such as the BIOS of a personal computer, may contain basic functions of a device, and may provide hardware abstraction services to higher-level software such as operating systems."

• Resolve host by request to DNS resolver
• Client SYN
• Server SYN+ACK
• Client SYN
• HTTP request
• HTTP response

False. It doesn't maintain state for incoming request.

• Request line - request type
• Headers - content info like length, enconding, etc.
• Body (not always included)

• GET
• POST
• HEAD
• PUT
• DELETE
• CONNECT
• OPTIONS
• TRACE