Python data science libraries that you should know

8 python libraries for data science

1. NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays.

In python interpreter

>>> import numpy as np
>>> a = np.arange(15).reshape(3,5)
>>> a
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14]])

2.scikit-learn - is an open source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection and evaluation, and many other utilities.features various algorithms like support vector machine, random forests, and k-neighbours, and it also supports Python numerical and scientific libraries like NumPy and SciPy.

>>> clf = RandomForestClassifier(random_state=0)
>>> x=[[1,2,3],
... [11,12,13]]
>>> y = [0,1]
>>> clf.fit(x,y)
RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,
                       criterion='gini', max_depth=None, max_features='auto',
                       max_leaf_nodes=None, max_samples=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=1, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, n_estimators=100,
                       n_jobs=None, oob_score=False, random_state=0, verbose=0,
                       warm_start=False)
>>> clf.predict(x)
array([0, 1])
>>> clf.predict([[4,5,6],[13,14,15]])
array([0, 1])

3.pandas
When working with tabular data, such as data stored in spreadsheets or databases, Pandas is the right tool for you. Pandas will help you to explore, clean and process your data. In Pandas, a data table is called a DataFrame.

https://pandas.pydata.org/getting_started.html

The primary two components of pandas are the Series and DataFrame

>>> import pandas as pd
>>> data = {
...     'naveen':[50,40,30,20],
...     'John':[23,50,34,22]
... }
>>> marks = pd.DataFrame(data)
>>> marks
   naveen  John
0      50    23
1      40    50
2      30    34
3      20    22
>>> marks = pd.DataFrame(data,index=['English','Maths','Science','History'])
>>> marks
         naveen  John
English      50    23
Maths        40    50
Science      30    34
History      20    22

4.Sympy -
SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible

>>> from sympy import solve,Eq,symbols
>>> x, y, z, d = symbols('x y z d') 
>>> eq1 = Eq(x+y,8) 
>>> eq2 = Eq(x+z,13)  
>>> eq3 = Eq(z+d,6)  
>>> eq3 = Eq(z-d,6)  
>>> eq4 = Eq(y+d,8)
>>> solve(eq1,eq2,eq3,eq4,(x,y,z,d))

5.mathplotlib -Matplotlib is a plotting library for the Python programming language and its numerical mathematics extension NumPy

>>> import matplotlib.pyplot as plt
>>> import numpy as nu
>>> x = nu.linspace(0,10,100)
>>> plt.plot(x,x,label='linear')
[<matplotlib.lines.Line2D object at 0x7fe300680910>]
>>> plt.legend()
<matplotlib.legend.Legend object at 0x7fe2f38f6450>
>>> plt.show()

6. Tensorflow
TensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks.

https://www.tensorflow.org/api_docs/python

https://machinelearningmastery.com/introduction-python-deep-learning-library-tensorflow/

7. Keras: The Python Deep Learning library
Keras is an open-source neural-network library written in Python. It is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, R, Theano, or PlaidML.
There are two main types of models available in Keras: the Sequential model, and the Model class used with the functional API.

https://keras.io/

8.Scipy -

https://www.scipy.org/getting-started.html

The SciPy ecosystem

Scientific computing in Python builds upon a small core of packages:

• Python, a general purpose programming language. It is interpreted and dynamically typed and is very well suited for interactive work and quick prototyping, while being powerful enough to write large applications in.

• NumPy, the fundamental package for numerical computation. It defines the numerical array and matrix types and basic operations on them.

• The SciPy library, a collection of numerical algorithms and domain-specific toolboxes, including signal processing, optimization, statistics, and much more.

• Matplotlib, a mature and popular plotting package that provides publication-quality 2-D plotting, as well as rudimentary 3-D plotting.

10 commands that you should know in Linux networking

1. ifconfig - configure a network interface

examples : -

- View network settings of an ethernet adapter:

ifconfig eth0



- Display details of all interfaces, including disabled interfaces:

ifconfig -a



- Disable eth0 interface:

ifconfig eth0 down



- Enable eth0 interface:

ifconfig eth0 up



- Assign IP address to eth0 interface:

ifconfig eth0 ip_address

2. traceroute - to trace route or path of the packets to the destination machine

examples :

- Traceroute to a host:

traceroute host

  

- Disable IP address and host name mapping:

traceroute -n host

  

- Specify wait time for response:

traceroute -w 0.5 host

  

- Specify number of queries per hop:

traceroute -q 5 host

  

- Specify size in bytes of probing packet:

traceroute host 42

  

3.telnet - The telnet command is used for interactive communication with another host using the TELNET protocol.

- Telnet to the default port of a host:

telnet host



- Telnet to a specific port of a host:

telnet ip_address port



- Exit a telnet session:

quit



- Emit the default escape character combination for terminating the session:

Ctrl + ]



- Start telnet with "x" as the session termination character:

telnet -e x ip_address port

4.nslookup - nslookup is a program to query Internet domain name servers.

- Query your system's default name server for an IP address (A record) of the domain:

nslookup example.com



- Query a given name server for a NS record of the domain:

nslookup -type=NS example.com 8.8.8.8



- Query for a reverse lookup (PTR record) of an IP address:

nslookup -type=PTR 54.240.162.118



- Query for ANY available records using TCP protocol:

nslookup -vc -type=ANY example.com



- Query a given name server for the whole zone file (zone transfer) of the domain using TCP protocol:

nslookup -vc -type=AXFR example.com name_server



- Query for a mail server (MX record) of the domain, showing details of the transaction:

nslookup -type=MX -debug example.com



- Query a given name server on a specific port number for a TXT record of the domain:

nslookup -port=port_number -type=TXT example.com name_server

5. netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships.

- List all ports:

netstat -a



- List all listening ports:

netstat -l



- List listening TCP ports:

netstat -t



- Display PID and program names:

netstat -p



- List information continuously:

netstat -c



- List routes and do not resolve IP to hostname:

netstat -rn



- List listening TCP and UDP ports (+ user and process if you're root):

netstat -lepunt



- Print the routing table:

netstat -nr

6. ip - show / manipulate routing, devices, policy routing and tunnels

- List interfaces with detailed info:

ip a



- Display the routing table:

ip r



- Show neighbors (ARP table):

ip n



- Make an interface up/down:

ip link set interface up/down



- Add/Delete an ip address to an interface:

ip addr add/del ip/mask dev interface



- Add a default route:

ip route add default via ip dev interface

7. nmap - Network exploration tool and security / port scanner

- Try to determine whether the specified hosts are up and what are their names:

nmap -sn ip_or_hostname optional_another_address

  

- Like above, but also run a default 1000-port TCP scan if host seems up:

nmap ip_or_hostname optional_another_address

  

- Also enable scripts, service detection, OS fingerprinting and traceroute:

nmap -A address_or_addresses

  

- Assume good network connection and speed up execution:

nmap -T4 address_or_addresses

  

- Scan a specific list of ports (use -p- for all ports 1-65535):

nmap -p port1,port2,…,portN address_or_addresses

  

- Perform TCP and UDP scanning (use -sU for UDP only, -sZ for SCTP, -sO for IP):

nmap -sSU address_or_addresses

  

- Perform TLS cipher scan against a host to determine supported ciphers and SSL/TLS protocols:

nmap --script ssl-enum-ciphers address_or_addresses -p 443

  

8.ping - send ICMP ECHO_REQUEST to network hosts

- Ping host:

ping host

  

- Ping a host only a specific number of times:

ping -c count host

  

- Ping host, specifying the interval in seconds between requests (default is 1 second):

ping -i seconds host

  

- Ping host without trying to lookup symbolic names for addresses:

ping -n host

  

- Ping host and ring the bell when a packet is received (if your terminal supports it):

ping -a host

  

- Also display a message if no response was received:

ping -O host

9.ip link - network device configuration

ip link add [ link DEVICE ] [ name ] NAME



ip link set { DEVICE | group GROUP } { up | down | arp { on | off } } etc...

10. docker network command - Manage networks. You can use subcommands to create, inspect, list, remove, prune, connect, and disconnect networks.

docker network connect  Connect a container to a network

docker network create  Create a network

docker network disconnect  Disconnect a container from a network

docker network inspect  Display detailed information on one or more networks

docker network ls  List networks

docker network prune  Remove all unused networks

docker network rm  Remove one or more networks

Understanding hashing and applications of hashing - Rust

Rust Hashing

I am seeing hashing everywhere like block chain, load balancing ,Cryptographic hash functions,Password Verification,key-value pair data structures in programming languages.

So thought of checking how to do hashing Rust and looks like its pretty easy to do in Rust as well.

#![allow(unused)]
fn main() {
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};

#[derive(Hash)]
struct Person {
    id: u32,
    name: String,
    phone: u64,
}

let person1 = Person {
    id: 5,
    name: "Janet".to_string(),
    phone: 555_666_7777,
};
let person2 = Person {
    id: 5,
    name: "Bob".to_string(),
    phone: 555_666_7777,
};

assert!(calculate_hash(&person1) != calculate_hash(&person2));

fn calculate_hash<T: Hash>(t: &T) -> u64 {
    let mut s = DefaultHasher::new();
    t.hash(&mut s);
    println!("{:?}",s.finish());
    s.finish()
}
}

hashing code example in rust