Saturday, February 5, 2022

May be Rust can speed Django projects ?

I was trying to plugin pyo3 build Rust '.so' files(dynamic libraries) to Django. Looks like its not that difficult, we can copy and paste the file to Django folder. Once the file avaliable in Django folder , it will allow us to import the Rust dynamic library functions to your Django views.

Step1: create the Rust program as library using pyo3 and build the file using cargo build. Remember to put the crate-type "cdylib". I used manual build and copy .so file . Remember to keep the python module name and .so file name as same. https://pyo3.rs/v0.15.1/
Step2 Create your normal Django helloworld App
Step3 Copy the .so file to your Django project folder and import it in your views.py file, then you should be able to run django server .

Saturday, March 27, 2021

Load tested my Django site

Benchmark HTTP Latency with wrk

 

Recently I started reading through Actix-web, Django,tokio etc, so thought of bench marking the latency. Started with Django as it is pretty easy get started and its in python.

 

looks like wrk ( https://github.com/wg/wrk )  is a good tool to start with. 

"wrk is a modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU. It combines a multithreaded design with scalable event notification systems such as epoll and kqueue."

Installation also pretty easy in Ubuntu:


1. sudo apt-get install build-essential libssl-dev git -y

2. git clone https://github.com/wg/wrk.git wrk

3. cd wrk/

4. sudo make

5. sudo cp wrk /usr/local/bin


Run your Django server using 


python manage.py runserver

 

then another terminal you can run your wrk as below

 

wrk -t12 -c400 -d30s --latency http://127.0.0.1:8000

 I only had single page and not much custom files in the Django and not data base request. Not sure this is a good response .. Please share your comments.



 Also got something like this Django logs as well.

[27/Mar/2021 17:39:14] "GET / HTTP/1.1" 200 196
[27/Mar/2021 17:39:14] "GET / HTTP/1.1" 200 196
Exception happened during processing of request from ('127.0.0.1', 33856)
[27/Mar/2021 17:39:14] "GET / HTTP/1.1" 200 196
[27/Mar/2021 17:39:14] "GET / HTTP/1.1" 200 196



I am planning to try this same experiment with actix-web and Rust , will share once I have something..

 

Wednesday, January 6, 2021

Yew App - Sudoku solver

How would be a Yew app Sudoku solver ?

sudoku solver can  basically be  a backtracking algorithm . Last month I implemented a Rat-Maze problem solver with Yew App. But it does not look visually much attractive to some of my friends. So thought of doing a sudoku solver with same code modifying the logic for sudoku.

Demo video 




if you are interested to see the code, it here ( Note: its an ugly code)

https://github.com/davnav/sudokuyewapp

Tuesday, December 29, 2020

Rat-Maze problem - Yew app - PART2

Rust Yew app:


Last week I wrote about the  Rat-Maze problem - Yew app.

Writing code  in Rust programming language is always fun and a learning.So thought of making some enhancements to the yew app 

1. add colors to the cells

2. after solving the path, change the color of the cells in the path. ..etc


This gave an opportunity to work with CSS and Yew. I think still there is no full support for CSS from Yew as you might have seen in React. But I was managed to change the color with "class" property in CSS.

index.css


.game-celluley {
display: inline-block;
width: 20px;
height: 20px;
border: 1px solid #ccc;
background-color: yellow;
text-align: center;

}


.game-celluleg {
display: inline-block;
width: 20px;
height: 20px;
border: 1px solid #ccc;
background-color: green;
text-align: center;

}



lib.rs

html!{
<div key=x class=format!("game-cellule{}",cell_color)>{ cell.value } </div>
}


Source Code:

github- Yew app


How to compile and run:




output:




Saturday, December 26, 2020

Rat-Maze problem - Yew app built with rustwasm

 Backtracking Algorithms

I just started loving the algorithms , especially backtracking algorithms.This geekforgeeks page gives more details on backtracking 

https://www.geeksforgeeks.org/rat-in-a-maze-backtracking-2/

Solving this problem in Rust programming  will be  a little more interesting. Even more fun if we can build this as  Rust Wasm app. Webassembly (wasm) is one of the latest technology that web developers are interested in. So thought of building  the rat-maze as a Yew app.

Yew is a modern Rust framework for creating multi-threaded front-end web apps with WebAssembly

Read mroe about Yew here - https://yew.rs/docs/en/


I used following dependencies - 

yew="0.17"
wasm-bindgen="0.2.67"
web-sys = "0.3.46"
wasm-logger = "0.2.0"
log = "0.4.6"


Backtracking -

This is one of the algorithm every programmer should try and understand. It will involve recursion and push and pop values from Results.

In my program, "solve" is the method actually implemented this backtracking

///Actual method where the backtracking algorithm is implemented
/// this method will be recursively called based the choices
/// we will be adding the "path" in this algorithm, before calling the method recursively
/// if we couldn't solve the Maze, pop the added "path" and next choice we have.


Yew Component:



///Grid is our component which displays the maze
/// Its also contains the path , once we solve it
struct Grid{
link:ComponentLink<Self>,
cell:[[u8;4];4],
path:Vec<(usize,usize)>,

}


The full program is below for your reference:

Full Code for Rat Maze


Output:




Wednesday, December 9, 2020

How to make dereference for your struct in Rust

 Dereference operator ( *)  in Rust


Dereference operator we have seen in C programming language. Yes , I am talking about the ' * ' operator that we use for dereferencing a pointer in C.

I am not sure how * operator works behind the seen, but the it dereference the pointer and give us the value in that address. 

But in Rust implementation is more obvious  that the type has to implement the trait 'Deref".

https://doc.rust-lang.org/std/ops/trait.Deref.html 


So let's see how should be convert our struct to dereference the value whatever way that we like.


Rust  Deref example

This is simple program to use * operator on MyStruct. And it is only to demonstrate the capabilities of Deref trait, not an idiomatic code.

Here we are derefencing the second field value when we use * operator on MyStruct instance.

use std::ops::Deref; struct MyStruct{ x: i32, y:String, } impl Deref for MyStruct{ //the Deref trait implementation should have Target //associated type type Target = String; // trait mandotory function implementation fn deref(&self) -> &Self::Target{ &self.y } } fn main(){ let my = MyStruct { x:4,y:"hello".to_string() }; //derefencing our struct with * operator println!("{}",*my); }



Conclusion:

Even though references and dereferencing are essential part of most of the system programming languages, how they are implemented  are always hidden behind the language implementations ( in my knowledge) . But Rust gives that insight and flexibility to use them on custom references.



Saturday, November 14, 2020

Callbacks in Rust and Python



Callbacks

Following is the definition of callbacks from wiki https://en.wikipedia.org/wiki/Callback_(computer_programming)

callback in Rust

Whenever I read any web code, callbacks are important part of the code.
So tried to build a basic callback in rust to understand what is it .


I believe , callbacks are basically storing functions/closures in array and calling them whenever required. Actually it is a little difficult to build in Rust because of the syntax and type checks.
But it’s work , type of the functions will be verified in compile time.
For example , if I give x+2 , instead of x +=2 in the below code, rust comiplier gives compilation error.
below code


Rust callback


//create a callbacks list and make some basic operations like
//register a function and calling it etc.

//callbacks struct to store the vector( list of functions)
struct CallbackV1{
    callbacks:Vec<Box<FnMut(i32)>>
}

//implemenation of the CallbackV1
impl CallbackV1{

    fn new() ->Self{

        CallbackV1 {
            callbacks : Vec::new()
        }
    }
	
	//method to register functions in the callbacks vector
    fn register(&mut self,f:Box<FnMut(i32)>){
        self.callbacks.push(f)
    }

	//method to call the functions in the callbacks
    fn call(&mut self,p:i32){

		//iterate over the callbacks vector and take a mutable reference
        for callback in self.callbacks.iter_mut(){
			
			//dereference the callback function and call with passed argument.
            (&mut *callback)(p);
        }
    }
}

fn add(x:i32){

    println!("{}",x+1);
}

fn main() {
//   let mut x = 2;
   let mut  mycallbacks = CallbackV1::new();
   let f1 = |mut x:i32| { 
                            x += 2
                            // println!("{}",x) ;
            };
   let f2 = add;

   mycallbacks.register(Box::new(f1));
   mycallbacks.register(Box::new(f2));

   mycallbacks.call(4);


}


Callback in Python

Built the same code with Python as well. Its quite easy to build the same in python. But the type checks are not there . Its simply call the functions registered. I believe this can lead to vulnerabilities .

Python Callback code


#create a callbacks list and make some basic operations like
#register a function and calling it etc.

class CallbacksV1:
    
    def __init__(self):
        self.callbacks = []
    
    def register(self,f):
        self.callbacks.append(f)
    
    def call(self,value):
        
        callback =  self.callbacks.pop()
            
        callback(value)

    
def add(x):
    print(x+1)

def add_two(x):
    print(x+2)
    return (x+2) 

c = CallbacksV1()
c.register(add)
c.register(add_two)

c.call(3)
c.call(5)

Saturday, July 25, 2020

Why tokio tasks are run-time dependent ?


Tokio and async-std 


I was going through two major crates for async programming in Rust. I believe Tokio and async-std are predominant crates now available for async programming. 

But when I tried to run tokio tasks with async-std run-time, but default it won't allow. I need to change run-time as tokio using #[tokio::main] that's not nice.

But when  I tried to run async-std tasks with tokio run-time, I am able to run.  Looks like async-std is more compatible with other Rust eco-systems.



error:
thread 'main' panicked at 'must be called from the context of Tokio runtime configured with either `basic_scheduler` or `threaded_scheduler


There would be libraries/crates to make it happen, but I think by default this should be available.


Friday, July 24, 2020

Where is the infinite loop in the below Rust Program ?

Have you ever wonder how the below  TcpListener Code run infinitely looking for new connections/clients.

Code:
When you run this below program with a Rust compiler or Cargo run, you will see that the program is not ending. How ?


As you noticed , the for loop doesn't have a explicit condition to exit. As far as the Iterator can return values, the "for loop" construct  execute the  statements inside the body. The Iterators can behave like  an infinite loop.But can return "None"(which can be used to exit from "for") , or can return Some values from next method .

Read more to understand how "for loop"  decompose in Rust - https://doc.rust-lang.org/reference/expressions/loop-expr.html

Ok..we understood that there is something called Iterator which can yield for values infinite.

But where is the Iterator here ?

When you look at the TcpListener , it is just a struct. But it has one particular method which implements Iterator trait. 
The connections can be accepted from TcpListener through a  struct called  std::net::Incoming  which is an Iterator.It is hidden(abstracted) behind the method incoming  in this case. As I said earlier  Iterators can yield and ask for values infinite times.





Iterator Trait is implemented on the struct "Incoming"



As you noticed the next method on the Iterator is not going to end. It just keep iterate as far as we are calling incoming method.




Incoming struct accepts connections of TcpListener:



Incoming implementing Iterator trait:




Saturday, July 18, 2020

compiled a Rust program for Wasm target without wasmbind-gen

It was not that hard to write a function and build for wasm target. I was using the #![no_std] without knowing much on it.The Rust compiler was just guiding through errors to fill the code.
I did not use the wasmbind-gen. But I did use wee_alloc

The program is below:

Note: just comment each line and see the error message compiler is giving to you.

Sunday, June 14, 2020

Pointers in Rust

pointers in Rust

Pointers

Pointers - what is the definition of pointer?
As per wikipedia - In computer science, a pointer is a programming language object that stores a memory address. This can be that of another value located in computer memory, or in some cases, that of memory-mapped computer hardware.

https://en.wikipedia.org/wiki/Pointer_(computer_programming)

Historically pointers become a tough topic for students, even though the definition looks simple. The loop holes behind the pointers make programmers life hard with them.
If I write a program in C with pointers, its end up in “Segmentation Fault”

Pointers in Rust

I have seen Computer languages started introducing smart pointers recently for safety . So since Rust is new language, Rust has really smart pointers .But it may looks a bit more syntax around them. Once you know the specification of them its quite safe to use them.

1.Raw pointers in Rust

Let starts with dirty one(it’s not dirty,just not smart). Raw, unsafe pointers, *const T, and *mut T.
As name suggests they are raw or unsafe pointers. You can assign a raw address to a raw pointer.

These pointers are very useful when you need to assign some port address to a variable in Rust.

fn main() {
    let p: *mut i32;
    p = 0x10 as *mut i32 ;
    
}

But if you try to deference and assign a value , it throws error in Rust. Rust error shows you why it is risky…

```rust_errors
|
5 |     *p= 10;
  |     ^^^^^^ dereference of raw pointer
  |
  = note: raw pointers may be NULL, dangling or unaligned; they can violate aliasing rules and cause data races: all of these are undefined behavior

But if you explicitly tell the compiler that I am sure this is safe to use and so am tagging this snippet under “unsafe” , the compiler compile the code successfully
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=792e09bd5485af4ceca96e59ba80848b

Read more: https://doc.rust-lang.org/std/primitive.pointer.html

2.references - &T

References are generally called as pointers. In Rust,&T type a ‘reference’. So rather than calling this as pointers ,Rust consider this as a separate Type.
&T means reference to a type value T. You can dereference it with * operator.

fn main() {
    let a = 10;
    let p = &a;
    println!("{}",*p);
    
}

3.References - &mut T

Since it is a Type , &T and &mut T are different. The first one is simple reference and we can access the value that points ,but can’t modify the value.
You can modify the value with &mut T reference
Rust treats different variation(mutable or immutable) of references as different types
eg:

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=776367d50a05cc1af60c90b31d8ac0d7

But mutable references have one big restriction: you can have only one mutable reference to a particular piece of data in a particular scope

The Rules of References

  • At any given time, you can have either one mutable reference or any number of immutable references.
  • References must always be valid.
fn main() {

    let mut a = 10;
    
    let p1 = &mut a;
    
    *p1 = 11;
   
    let p2 = &a;
    
    println!("{},{}",*p1,*p2)
    
}

Above code won’t compile with error message

rust_errors
error[E0502]: cannot borrow `a` as immutable because it is also borrowed as mutable
  --> src/main.rs:9:14
   |
5  |     let p1 = &mut a;
   |              ------ mutable borrow occurs here
...
9  |     let p2 = &a;
   |              ^^ immutable borrow occurs here
10 |     
11 |     println!("{},{}",*p1,*p2)
   |                      --- mutable borrow later used here

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1a975610b3d5d380e57dea9cc1c286b4

If you use 2 mutable reference, the program won’t compile successfully.

fn main() {

    let mut a = 10;
    
    let p1 = &mut a;
    
    *p1 = 11;
   
    let p2 = &mut a;
    
    println!("{},{}",*p1,*p2)
    
}

error message is below, hope that is self explanatory.

error[E0499]: cannot borrow `a` as mutable more than once at a time
  --> src/main.rs:9:14
   |
5  |     let p1 = &mut a;
   |              ------ first mutable borrow occurs here

9  |     let p2 = &mut a;
   |              ^^^^^^ second mutable borrow occurs here
10 |     
11 |     println!("{},{}",*p1,*p2)
   |                      --- first borrow later used here

4.Box - A pointer type for heap allocation.

A Box allows heap allocations similar to “malloc” in C programming.
You can mutate the heap value using the keyword ‘mut’ in front of the variables.

fn main() {

   let mut heap_pointer = Box::new(10);
    *heap_pointer += 1;
   println!("{}",*heap_pointer);
   
}

The above code gives you the output as 11.

 fn main() {

    let mut heap_pointer = Box::new(10);
    
    let heap_pointer2 = &mut heap_pointer ;

   // let heap_pointer3 = &mut heap_pointer;
   // heap_pointer2 = *heap_pointer2 + 1;
   
   match *heap_pointer {
       x => { println!("{}",x+34)},
   }
    
 //   let heap_pointer3 = &mut heap_pointer;
  //   *heap_pointer2 = *heap_pointer + 1;
    println!("{}",*heap_pointer);
 //   println!("{}",x);
 //   println!("{}",*heap_pointer2);
    
}

How to get the value from Box:
You can use * operator to dereference the address.

#![allow(unused)]
fn main() {
let x = Box::new(String::from("Hello"));
println!("{}",*x);
}

Using Raw pointer -

You have to use “unsafe” block for creating a raw pointer for a boxed value.

#![allow(unused)]
fn main() {
let x = Box::new(String::from("Hello"));

//taking a raw pointer
let ptr = Box::into_raw(x);

let x = unsafe { Box::from_raw(ptr) };

println!("{}",x);
}

5.Rc - Reference Counted

We are getting into the world of smart pointers. Reference counting is not a new concept, it is there in Python. If you see below program in python tutor.

a = [12,3,4]
b = a
c = b

a = 0
b= 1
c =2

The python visualizer shows you that the object (list object) remains in the memory until all the references to the object get removed.
Note: when we assign a =0 , the value assigned to ‘a’ become zero .

http://www.pythontutor.com/visualize.html#code=a %3D [12,3,4] b %3D a c %3D b a %3D 0 b%3D 1 c %3D2 &cumulative=false&curInstr=6&heapPrimitives=nevernest&mode=display&origin=opt-frontend.js&py=3&rawInputLstJSON=[]&textReferences=false

In Rust ‘Rc’ is a single-threaded reference-counting pointer. ‘Rc’ stands for ‘Reference Counted’.

Below program creates an Rc reference and get the value using * operator (dereferencing )



fn main() {
use std::rc::Rc;

let x = Rc::new("hello".to_owned());

let y = Rc::clone(&x);
let z = Rc::clone(&x);

println!("{}",*x);
println!("{}",*y);

println!("{}",Rc::strong_count(&y));

}

Below Rust program generates number of references from a heap allocated memory . You might have noticed that we drop ‘x’ in the middle. But still we have 2 Rc references. If you have multiple normal reference &T for a heap allocation, compiler won’t allow you to drop the owned reference.

fn main() {
use std::rc::Rc;
let x = Rc::new("hello".to_owned());

let y = Rc::clone(&x);
let z = Rc::clone(&x);

drop(x);

println!("{}",Rc::strong_count(&x));
}

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=12748f50761fb40ae36bead095f64a65

6.Recell

A mutable memory location with dynamically checked borrow rules.
Refcell gives you a mutable reference for an immutable instance.
if you do two or more ‘borrow_mut’ in the same scope, the program panic in the run time.

But below program , we are passing borrow_mut to the function, so it is not in the same scope.

#![allow(unused)]
use std::cell::RefCell;
use std::cell::*;

fn main() {

//c is not declared as mutable.
let  c = RefCell::new(5);

//but we are creating a mutable borrow and passing to the function.
fun (c.borrow_mut());
fun (c.borrow_mut());

println!("{}",c.into_inner());

}

//fun function receive a mutable cell
fn fun(mut x:RefMut<i32>){
    
// changing the value inside the cell, even though cell 'c' was not declared as mutable.
    *x = 3;
}

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=02b0ed5116b1abada7160c6ee0fbb436

You can wrap RefCell(not declared mutable) with Rc and mutate the object with Rc reference . If you are using immutable Rc for another objects other than RefCell, it won’t allow to mutate .

#![
allow(unused)]
use std::cell::RefCell;
use std::cell::*;
use std::rc::Rc;
fn main() {

//c is not declared as mutable.
let  c = RefCell::new(5);

//wrap Refcell with Rc

let r = Rc::new(&c);
//wrap Refcell with Rc, even though r1 is not mutable , we use it mutating cell value
let r1 = Rc::clone(&r);

let r2 = Rc::clone(&r);



//but we are creating a mutable borrow and passing to the function.
fun (r1.borrow_mut());

//let d = c.borrow();
fun( r2.borrow_mut());


println!("{}",c.into_inner());

}

//fun function receive a mutable cell
fn fun(mut x:RefMut<i32>){
    
   // changing the value inside the cell, even though cell 'c' was not declared as mutable.
    *x += 3;
} 

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=75d5ccbb4c6c9d62d5df4a92a8f7137e

Conclusion

You might need to try these pointers and write some small code snippets and see to understand how Rust compiler behaves on these pointers. The smart pointers really safe to use in Rust. You can avoid bugs that are hard to debug like race condition, double free,free after use etc…

This is just my notes and experiments with Rust pointers.There could be some missing or incorrect information. Happy to correct it !!

Monday, June 8, 2020

Rust - Test driven program development

Cargo supports test driven developmentin Rust programming. There are lot of options comes with cargo test command.


Writing test functions with #[test] in your program makes the code more readable as well as its makes easy to understand the functionalities of the program for others . I have seen that even I can start running the complex projects and see outputs of some of the modules or functions in those projects.
code repository - https://github.com/davnav/parser_try.git