Tuesday, March 31, 2020

How do C and Rust programs differs in memory-safety - example 1

ownership

How do C and Rust differs in memory safety ? example 1

Looks at the below program, how crazy the ‘main’ function snatches the password from the function ‘assign’. We only return the pointer to the ‘user’, but ‘main’ gets ‘password’ from it.

It took some trials for me to find number nine(9) as the offset (byte) ( difference between the “user” and " password" address ) .

C code

 l 1 
  2 #include<stdlib.h>
  3 
  4 char* assign(){
  5 
  6     char password = 'a'; //password stored here
  7     char b[3] = "ab";
  8 
  9     int* username = &b;
 10 
 11     return username;
 12 }
 13 
 14 
 15 
 16 int main(){
 17 
 18   char* user = NULL;
 19 
 20   user  = assign();
 
 21   // you can just do some address offseting with "-" or "+" to get the "password" field
 
 22   printf("%c\n",*(user - 9));
 23 
 24 }

Output is ‘a’ which is the password here.

a

Rust code:

In rust it is very difficult or not even possible ( I don’t know a method) without unsafe code to offset the address and get another variables or string slices(literals).

Interestingly “password” and “username” variables are referring to the program memory itself ( not stack or heap ) as &str ( string slice) hard coding in program binary.

We can try with "unsafe " code to do some manipulation on the username address to snatch the “password”.

But if you are making sure that no ‘unsafe’ code in your program , you can avoid this scenario in rust or you will catch those scenario on compile time itself ( that is AWESOME! )

 2 fn assign() -> &'static str{
  3 
  4     let password = "q";
  5     let username = "asdasd";
  6 
  7     let q = username;
  8 
  9 
 10     println!("pointer = {:p}, {:p}",password,q);
 11 
 12     q
 13 
 14 }
 15 
 16 
 17 fn main() {
 18 
 19   let p:&str = assign();
 20 
 21   let ptr: *const u8 = p.as_ptr();
 22 
   /// you really need to write unsafe code and
   /// do some trick to get offset address of the 
   /// "password" here. The + , - operators won't work with address in rust.
   
 23   unsafe {
 24           println!("Hello, world! : {}",*ptr.sub(1) as char);
 25   }
 26 
 27 }

          

Output is

pointer = 0x5638e3aa3d70, 0x5638e3aa3d71
Hello, world! : q

Sunday, March 29, 2020

Rust Traits Must know points

Reading and understanding Traits is a pre-requisite before starting to read any code base in Rust.

Most of the scenarios and examples are mentioned here,

https://doc.rust-lang.org/1.8.0/book/traits.html

Tuesday, March 24, 2020

Rust iterator types

I was  reading this .. honestly I lost in between.. https://blog.jcoglan.com/2019/04/22/generic-returns-in-rust/

Understanding traits and type bounds  are hard in rust , amazing fact is that  Rust is doing this with zero cost.

Thursday, March 19, 2020

Python Malayalam Book

I started long back to write Python programming helper book in Malayalam.  But unfortunately I got side tracked and haven't touch that for long time.

https://davnav.github.io/MalayalamPythonebook/first%20steps


Now again I am brushing it up .  Yesterday I moved that to github and converted to markdown files.

https://github.com/davnav/MalayalamPythonebook/

I can do lot of  md formatting now.

Hopefully I make some progress in coming days..

Looking for your support as well.





Wednesday, March 18, 2020

Passing function as parameter in Rust

Passing function as argument in Rust

Passing function type check in Rust

Passing a function to another function is not a new thing in programming. We usually pass the address of the function( in C like languages &function name ).
but do we ensure that passed function performing the intended functionality or at least the parameter and return types are matching with what we intended to pass.

Rust asks passed function signature

Rust explicitly ask for the type of the pass function signature . If it’s not matching the rust program won’t compile.
that means - no one can inject anonymous functions to our function for some extend.

Rust passed function

In the below example , you would notice an extra type in the function signature. Anything in ‘’<>" are generic type in Rust. But what is that generic type means ?
If you don't know the type of the function parameter, you can specify it as generic.
It represent the passed function or closure type, that will be declared in the “where” clause.
P is a function type with signature Fn(i32) -> bool means it a closure or function which can accept an integer parameter and return a bool type value.
fn foo(x:i32,mult:P) -> i32
where P: Fn(i32) -> bool
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=b1a66f49cc31c9176d894ddd42c4422b
from the main function we call foo as below
foo(220,mult1); which is a valid call because the ‘‘mult1’’ function signature matches with generic type
///foo function definition - it has 2 parameter 
/// an integer value and a passed function
///return type of the function is integer
fn foo(x:i32,mult:P) -> i32
    /// passed function type  
    where P: Fn(i32) -> bool {
  
    ///calling the passed function and getting the return value
    let y = mult(x);
    
    ///some calculation around based on the value 'y'
    if y {
        x
    }else{
        x - 1
    }
}
///the signature of this function matches with foo's passed function signature 'P'
fn mult1(x:i32) -> bool {
    x> 32
}
///the signature of this function not matches with foo's passed function signature 'P'
///as it return integer
fn mult2(x:i32) -> i32{
  x
}

fn main() {
    ///call to foo with passed function 'mult1'
    let q = foo(220,mult1);
    println!("Hello, world! = {}",q);   
}
When I change the same program to call foo with passed function as mult2,
foo(220,mult2);
got the below error message while compiling
error[E0271]: type mismatch resolving ` i32 {mult2} as std::ops::FnOnce<(i32,)>>::Output == bool`
  --> src/main.rs:30:13
   |
2  | fn foo(x:i32,mult:P) -> i32
   |    ---
3  |  
4  |   where P: Fn(i32) -> bool {
   |                       ---- required by this bound in `foo`
...
30 |     let q = foo(220,mult2);
   |             ^^^ expected `bool`, found `i32`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0271`.
error: could not compile `funexp1`.

To learn more, run the command again with --verbose.
the error clearly states below, which is awesome!!
where P: Fn(i32) -> bool {
   |                       ---- required by this bound in `foo`

Saturday, March 14, 2020

How do we make a C program call Rust program

C to Rust.md

Internals of C program call to rust program call

One of the main strength of Rust programming language is that it can easily inter-operate with other programming languages.

But as we know Rust is very very strongly typed language. When you are planning Rust to use some of the C libraries , the C type representation attributes help us .

#[repr( C )]

There are some difference in the type representation for C type corresponding rust type.

We can go-ahead and check what is the size of the below struct
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=7392513792a8d390ce4756a6a8a0ed15

use std::mem;
#[repr(C)]
struct FieldStruct {
    first: u8,
    second: u16,
    third: u8
}

// The size of the first field is 1, so add 1 to the size. Size is 1.
// The alignment of the second field is 2, so add 1 to the size for padding. Size is 2.
// The size of the second field is 2, so add 2 to the size. Size is 4.
// The alignment of the third field is 1, so add 0 to the size for padding. Size is 4.
// The size of the third field is 1, so add 1 to the size. Size is 5.
// Finally, the alignment of the struct is 2 (because the largest alignment amongst its
// fields is 2), so add 1 to the size for padding. Size is 6.
assert_eq!(6, mem::size_of::<FieldStruct>());
}

But if you remove #[repr( c )] , the struct size becomes 4.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f1ad5c57ae7e771ca505058784580a1e

#![allow(unused)]
fn main() {
use std::mem;
struct FieldStruct {
    first: u8,
    second: u16,
    third: u8
}
assert_eq!(6, mem::size_of::<FieldStruct>());
}

How to use a Rust function in C program.

I referred the blog for Sergey Potapov for the details.

In this below program, you would see the function print_hello_from_rust defined with extern keyword and [no_mangle] attribute.
[no_mangle] makes the compiler ignores the unknown symbols and it knows this function is going to get called from other languages.
extern keyword makes the function outside of the our library.

std::ffi::CStr. Representation of a borrowed C string. This type represents a borrowed reference to a nul-terminated array of bytes. It can be constructed safely from a &[ u8 ] slice, or unsafely from a raw *const c_char

In this example we are using *const c_char , but what is c_char ?!

c_char is coming from the standard library ‘os’ module and it is Equivalent to C’s char type.C’s char type is completely unlike Rust’s char type; while Rust’s type represents a unicode scalar value, C’s char type is just an ordinary integer. This type will always be either i8 or u8, as the type is defined as being one byte long

reference : c_char

then why *const in front of it ?

*const are called Raw pointers in Rust.Sometimes, when writing certain kinds of libraries, you’ll need to get around Rust’s safety guarantees for some reason. In this case, you can use raw pointers to implement your library, while exposing a safe interface for your users. Ref: Raw pointers

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=40d05db968f82125fb7660e67710ecff

use std::ffi::{CString,CStr};
use std::os::raw::{c_char,c_int};

#[repr(C)]
pub struct Point{
    x: c_int,
    y: c_int,
}

impl Point {

    fn new(x:c_int,y:c_int) -> Point{
//          println!("Creating a Point with x = {},y = {}",x,y);
          Point {x: x,y: y }

    }
}
#[no_mangle]
pub extern fn create_point(x:c_int,y:c_int) -> *mut Point{

        Box::into_raw(Box::new(Point::new(x,y)))
}
#[no_mangle]
pub extern fn print_hello_from_rust(data: *const c_char ){

    unsafe{
          let c_str =       CStr::from_ptr(data);
          println!("hello from rust {:?}",c_str.to_str().unwrap());
    }
}

You can build the rust program with cargo

cargo new whatland -- lib
cd whatland

edit the lib.rs file with above code.

also make sure that your cargo file has

 [lib]
 name = "whatland"
 crate-type = ["staticlib","cdylib"]

cdylib helps - A dynamic system library creation. This is used when compiling a dynamic library to be loaded from another language. This output type will create *.so files on Linux, *.dylib files on macOS, and *.dll files on Windows.

cargo build --release

C program would need to compile with linking the .so file generated.

I am not detailing the C program compilation, but it detailed in the blogpost: https://www.greyblake.com/blog/2017-08-10-exposing-rust-library-to-c/

You would need to use,

gcc -o ./examples/hello ./examples/hello.c -Isrc  -L. -l:target/release/libwhatlang.so

Passing String parameter from C to Rust function.

We can use the trick of accepting the string as a Raw pointer using *const c_char
In order to print that bytes into a valid string slice in Rust, first we need to convert that to CStr - Representation of a borrowed C string.

then we can convert that to Str ( rust string slice) using c_str.to_str().unwrap()

How to use a C struct in Rust program

I believe , Its always recommended to use #[repr©] when working with C structs, enums because it makes alignment https://doc.rust-lang.org/reference/type-layout.html#the-c-representation

#[repr(C)]
pub struct Point{
   x: c_int,
   y: c_int,
  
  //    x:u8, -- if you are using u8, compilier throws error,
  //    y:u8,    stating "expected `u8`, found `i32'"
  }

We implemented a method new in the rust program so that we can use in the Point struct instance creation.

impl Point {

  fn new(x:c_int,y:c_int) -> Point{

 Point {x: x,y: y }
 }
}

Next in this example we have a export function which will be called from C program to creating Point Struct. For creating any object, we need to memory. In Rust we know that we can allocate heap memory through Box::new.

But we need to return a raw pointer to the C program, which can be done through Box::into_raw which Consumes the Box, returning a wrapped raw pointer.

 23 #[no_mangle]
 24 pub extern fn create_point(x:c_int,y:c_int) -> *mut Point{
 25 
 26         Box::into_raw(Box::new(Point::new(x,y)))
 27 }

C program can now just call the Rust function to create struct.

We need to have our C program included with struct and create function declaration.
header file whatland.h

  1 void print_hello_from_rust();
  2 
  3 typedef struct Point{
  4             int x,y;
  5 }Point;
  6 
  7 Point* create_point(int x, int y);
  

Now we can just call it in our main function in C

 11  Point* p1 = create_point(10,20);
 12     printf("Point={%d},{%d}",p1 -> x,p1 -> y);

Monday, March 9, 2020

Self learn to write a File read program in Rust

cargo-test_debugging

Write a program to read a File in Rust

When I got this question first time, I don’t know where to start with this in Rust.
If this question was to write the same prgoram in C :

C is a school taught language and we know that C 's stdio has fopen, fclose ,fgets etc … because of that I wouldn’t have worried what/how fgets read or type conversion challenges are handled.

Here in Rust , I don’t which libraries/modules needs to imported
libraries are called crates in rust.
Some of the questions came to mind where

  1. what are all the standard libraries needed for this ?
  2. is there any method called ‘read’ in Rust ?
  3. how to get the file descriptor , is there any File Open ?

Looks like all these details are well documented in
std lib in rust

Finally I just started writing the program with whatever read method that I found somewhat relevant

  1 use std::fs::File;
  2 
  3 fn main() {
  4 
  5     let f = File::open("a.txt");
  6 
  7     match f.read() {
  8           Ok(x) => { println!("file contents = {}",x) },
  9           Err(e) => { println!("Error") },
 10     }
 11 }

Oops Error…
compiler complaints …

 error[E0599]: no method named `read` found for enum `std::result::Result<std::fs::File, std::io::Error>` in the current scope
 --> src/main.rs:7:13
  |
7 |     match f.read() {
  |             ^^^^ method not found in `std::result::Result<std::fs::File, std::io::Error>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0599`.
error: could not compile `std_learning1`.

Ok. read method is not there in fs inmodule.
We will goahead search where is the read in rust lang documentation.

I used std::fs::read , but if we read through the documentation carefully , we will understand that this is for small files and that can be parsed to a string format type like SocketAddr.

enter image description here

9   |     match std::fs::read(f) {
    |                         ^ the trait `std::convert::AsRef<std::path::Path>` is not implemented for `std::result::Result<std::fs::File, std::io::Error>

This error actually gives a clue that our “f” file object is “std::result::Result<std::fs::File, std::io::Error>”

I know that I can handles the error types with just adding “?” to file open.

But still we haven’t got which read method to use . Again going to back to documentation and search - I found a read that reasonable choice which is std::io:: Read::read

enter image description here

(Note:wrongly highlighted above)

just brought std::io::Read trait alone to the program for now.

as the function signature says -
fn read(&mut self, buf: &mut [u8]) -> Result<usize >

 1 use std::fs::File;
 2 use std::io::Read;
 4 
 5 fn main()  {
 6 
 7     let mut f = File::open("a.txt")?;
 8     let mut buf = [0;30];
 9     let n =  f.read(&mut buf[..]);
 
14 
15     println!("{:?}",&buf[0..n]);
16 
18 }


mainly two errors

  | |
7  | |     let mut f = File::open("a.txt")?;
   | |                 ^^^^^^^^^^^^^^^^^^^^ cannot use the `?` operator in a function that returns `()`
8  | |     let mut buf = [0;30];
error[E0308]: mismatched types
  --> src/main.rs:15:29
   |
15 |     println!("{:?}",&buf[0..n]);
   |                             ^ expected integer, found enum `std::result::Result`
   |
   = note: expected type `{integer}`
              found enum `std::result::Result<usize, std::io::Error>`

error: aborting due to 2 previous errors

we need to give return type in the main() function as we used “?” in line 7 and also we need to return Ok(()) in the main

We need to add io::Result<()> as return type in the main function, so for that we would need to include std::io;

So code would looks something like below:

  1 use std::fs::File;
  2 use std::io::Read;
  3 
  4 use std::io;
  5 //use std::io::prelude::*;
  6 
  7 fn main() -> io::Result<()>  {
  8 
  9     let mut f = File::open("a.txt")?;
 10     let mut buf = [0;30];
 11     let n =  f.read(&mut buf[..])?;
  17     println!("{:?}",&buf[0..n]);
 18 
 19    Ok(())
 20 }



But the output is still bytes.

   Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `/home/naveen/rustprojects/mar2020/std_learning1/target/debug/std_learning1`
[84, 104, 105, 115, 32, 105, 115, 32, 102, 105, 114, 115, 116, 32, 82, 117, 115, 116, 32, 115, 116, 100, 32, 108, 105, 98, 114, 97, 114, 121]

We can convert the bytes to string type using below statements.


 println!("{:?}",String::from_utf8((&buf[0..n]).to_vec()).unwrap());

One you might have noticed is that we need to handle the buffer explicitly.

So let us look again for some other read method available.

The one I found interesting is BufRead Trait which is a type reader that handle internal buffer.

Let us search for what are all the methods implementing this.

enter image description here

But how it can be used or related to File struct.

We know that File implements Read trait.

enter image description here

BufReader implements Read
BufReader implements BufRead as well.
So we can go-ahead are use/create an instance of BufReader on a File object.

Once we convert File object to an instance of BufReader , we can use methods like
lines
read_lines
read_until
split

  1 use std::fs::File;
  2 //use std::io::Read;
  3 
  4 use std::io::{self,BufReader};
  5 
  6 //include io  prelude which import all important structs and implementation
  7 //in this case it import all supporting BufReader implementation ( eg :  //BufRead)  for File struct.
  9 
 10 
 11 use std::io::prelude::*;
 12 
 13 fn main() -> io::Result<()>  {
 14 
 15     let mut f = File::open("a.txt")?;
 16 //    let mut buf = [0;30];
 17 
 18     let buf = BufReader::new(f);
 19 
 20  //   let n =  f.read(&mut buf[..])?;
 21 
 22    for line in buf.lines(){
 23         println!("{:?}",line);
 24    }
 25 
 26 
 27    Ok(())
 28 }



Diagram might be not fully correct. But I am trying to picturize the File read program modules and important internals.

Some notes in the above snippet:

  1. include io::prelude which import all important structs and implementation. in this case it import all supporting BufReader implementation ( eg : BufRead) for File struct.

  2. commented some of the lines(not deleted) which we used prior version of the program to understand the difference.

Output :

Ok("This is first Rust std library learning program.")

if we do an unwrap(), we will get the line string itself.

Conclusion:
Hope this helps you to understand how different traits are connected together at least for Rust File read program and which std modules needs to be imported for it. This same module/trait analysis required when you are working with external crates. It is important reading through the crate documentation and understand which traits are matching your requirement and which are needs to be imported and ready to use for their methods.

Saturday, March 7, 2020

regex programs should be a must tool for a programmer

Rust Regex:

I haven't given much attention to pattern matching. but I think it should be must taught topic for every programming student. having said that,  we can get into the my example



we can get into the Rust regex crate https://crates.io/crates/regex
and https://docs.rs/regex/1.3.4/src/regex/lib.rs.html#1-785





Since Rust handles string pretty nicely, regex expression matching is quite easy even if for my native language - Malayalam

you can either create an Rust regex object with Regex::new(r"(മല)") or RegexSetBuilder::new(&[r"à´œാà´µ"]).case_insensitive(true) .build()?;


if you really want to check each malayalam character , then you can use Regex::new(r"(p\{Malayalam}).unwrap()

For more detail how to use regex : 

https://www.regular-expressions.info/brackets.html

https://medium.com/factory-mind/regex-tutorial-a-simple-cheatsheet-by-examples-649dc1c3f285


Code details: regex malayalam search .

Thursday, February 20, 2020

gdb tips - PART1

gdb

Some gdb ( GNU debugger) tips

It is very important to understand the debugging tools and their different functionalities to hack the system level programs easily.

How to compile a program to debug in gdb.

  1. compile your C program with -g option

    $cc -g helloworld.c

  2. Open the executable in gdb
    $gdb a.out

  3. Run the program using below commands in gdb prompt using simply ‘r’.
    (gdb)run or (gdb)r

  4. Continue run from breakpoint using ‘c’
    (gdb)c

Setting Breakpoint in gdb.

1.Setting up breakpoint in a line number
$b 10
2. breakpoint setting up in an address
$b *0x7c00

(gdb) b *0x7c00
Breakpoint 1 at 0x7c00
(gdb) c
Continuing.
Thread 1 hit Breakpoint 1, 0x00007c00 in ?? ()

3.Setting up breakpoint in a function
$b main

Switch to tui mode using ctrl+x 2

you can switch to tui ( text gui mode) mode using ctrl+x 2
to view the source file in the same screen.
tui mode

you can ctrl+x 1 to see registers values

ctrl+x n to come back to gdb prompt.

stepping through line and instruction

$si \ steps through instruction
$s \ source level step through next line

Next step over through line

$n
$next

gdb_steps_through_source

Printing variables

$p foo//print the value of foo
$p car // print the value of car variable

$p/x addr //printing hexa decimal values

information about frames and registers

Current function stack frame details can be displayed using the command ‘f’ or ‘frame’
$f
$frame

enter image description here

The registers details or values can be seen using the command ’ info r’

info r
enter image description here

backtrace of call stacks

How can we backtrace all call stacks in gdb?!. This would be very useful command especially when you are debugging big projects source codes.

$bt
shows the backtrace of all call stacks so far.

enter image description here

You can switch to different frames using below command

$f 1
to switch to the first frame
f 5 switch to 5th frame etc

Some of the helpful commands when you are in a function

$info locals
shows the local variable in a function

$info args
shows arguments of the function

getting help on gdb

(gdb) help

List of classes of commands:

aliases – Aliases of other commands
breakpoints – Making program stop at certain points
data – Examining data
files – Specifying and examining files
internals – Maintenance commands
obscure – Obscure features
running – Running the program

(gdb)h breakpoints //provide help on breakpoints

PART 2 -
https://naveendavisv.blogspot.com/2020/04/gdb-debugging-part-2.html

Tuesday, February 18, 2020

Have you ever tried Ctrl+x 2 in GDB ?

I got amazed when I tried Ctrl+x  2 in gdb and step through code. I never thought GDB had an option to do this.





again press ctrl+x 2  .. you will see another magic.

Wednesday, February 12, 2020

Going to back OS/system related computer courses

I tried to read linux kernel and dropped it long back around the year of 2007 or 2008. Whenever I start touching system level code, programming languages becomes alien language for me. I thought of watching again the OS/computer architecture video courses. Luckily come across xv6 (https://en.wikipedia.org/wiki/Xv6)  operating system. This makes/motivating me to give one more attempt to learn the kernel and system level programming.

This one also helps me a lot.
  https://github.com/Babtsov/jos/tree/master/lab1

Wednesday, January 8, 2020

Emscripten - C programs to browser

--- ---

C programs and libraries to Browser

How nice it would be if I can call my own created C programs especially the computational intensive C programs from my Mozilla or Google Browser ( :-) in fact I don’t have anything that much computational intensive), But still . I would also be amazed if I can port some of the C libraries or C++ Games to browser.

Emscripten and Webassembly(wasm) together make this happen

I was following web-assembly and Rust for a few months now. Since Rust has lot of safety features, I did not think of exploring more on porting C or C++ programs to browser.

Emscripten is an Open Source LLVM to JavaScript compiler. Using Emscripten you can:
Compile C and C++ code into JavaScript
Compile any other code that can be translated into LLVM bitcode into JavaScript.
Compile the C/C++ runtimes of other languages into JavaScript, and then run code in those other languages in an indirect way (this has been done for Python and Lua)!

WebAssembly (often shortened to Wasm) is an open standard that defines a portable binary code format for executable programs, and a corresponding textual assembly language, as well as interfaces for facilitating interactions between such programs and their host environment.



Emscripten Installation


Emcripten is not that complex to install,there is good documentation in the site - Emscripten
You might need to install python2.7 and cmake etc as a prior requisite.We just need to follow the instructions. My Linux machine helped me to install everything with apt-get insall command.
You might come across some installation errors, but just search those errors in google, there will be solution to fix it.


C program to wasm


I wrote a simple C program called emccwasm.c
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<stdint.h>
int main(){
        int a = 1;
        printf("%d",a);
        //string function to put a space in the screen
        puts(" ");
        printf("First C program compiled to browser. a=%d",a);
        return 1;
}

In command line, you might need to use below command to compile if you are using Linux machine.
emcc -s NO_EXIT_RUNTIME=1 -s FORCE_FILESYSTEM=1 -s EXPORTED_FUNCTIONS=[’_main’] -o emccwasm.html emccwasm.c
Emscripten FAQ might help you understand these options and resolve the errors you might encounter quickly.
The above command generated 3 files for me , they are emccwasm.html , emccwasm.js and emccwasm.wasm


Emscripten generated Files.


Emcripten created a compiled webassembly file which is emccwasm.wasm as mentioned earlier.Filename will be same as the C program file name. To know more about webassembly , following documentation link helps - https://webassembly.org/
In a nutshell “WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server application
The .js and .html files are glue files created for web. The html file can be rendered to the browser and Javascript file initialize webassembly and rest of the glue work


http server


You might need to install http server to void below error . The following documentation can be used for installation -
https://github.com/thecoshman/http
Note: if you have installed Cargo in your machine, the server installation will be very easy.
"**Failed to execute ‘compile’ on ‘WebAssembly’: Incorrect response MIME type. Expected ‘application/wasm’.
**
We can run the server simply by below command:


http

Emscripten files in localhost


Once the http server is up, the files will be served in the browser in the link: http://localhost:8000/emccwasm.html
Note: the port might vary .
If everything goes well, you should be getting a screen similar to below.

enter image description here


Accessing exported functions.


When we compiled the C program we used EXPORT_FUNCTIONS option with _main. That means the _main function should be available in Module.
ie , we can invoke main function by Module._main()
enter image description here

Thursday, September 19, 2019

A bit research on String type in Rust

string

How did the String Type in Rust define?

There are two type of string types in Rust. Once is &str and other one is “String”.

String is heap allocated, grow able and not null termiated type.
&str is a slice(&[u8] ) that always points to a UTF8 valid sequence.
learn more about type usage here String types

But we are trying to focus on “String” type and how do we search Rust source code to see how this type is implemented.

String type - Heap allocation

Heap allocated memory implies that there will be pointer behind the definition of this type.

Let’s see,

We can get the rust source from below link.

https://doc.rust-lang.org/std/string/struct.String.html

click on the src button, as highlighted in the link.

enter image description here

it will directly take you to the surprising fact that String is simply a vector ( vec ).
String is a Vec<u8>

enter image description here

We know that Vec is also a heap allocated memory. In order to find how “vec” is defined in the source ,we need to search keyword “Vec” in the source code.

enter image description here

click on “src”
enter image description here

Interestingly, you will see the code.

pub struct Vec<T> {
    buf: RawVec<T>,
    len: usize,
}

I haven’t heard about RawVec earlier, but this is coming from : crate::raw_vec::RawVec;

raw-vec is implemeted for handling contiguous heap allocations.

enter image description here

Rawvec is implemented as below using a Unique pointer.

As guessed , we come to know that String type is actually using a pointer.

enter image description here

But what is unique pointer?
It comes from use core::ptr::Unique;

Unique is wapper around a raw non nullable ‘*mut T’.
interesting …

enter image description here

NonZero is wrapper type for raw pointers and integers that never becomes zero or null .

Tuesday, August 27, 2019

FFI tricks in Rust programming Language

Welcome file

How to do a trick on your Rust compiler to allow external (Foreign Language - FFI) functions

Rust is cool programming language which you can easily plugin to other languages ( Foreign Languages) .

Some of the FFI (Foreign language Interface) methods to call rust programs from C programming and Python programming languages are explained here.

Rust - Two things you should know for allowing Foreign language to call

#[no_mangle]
You might see lot of libraries using #[no_mangle] procedural macro.
The #[no_mangle] inform the Rust compiler that this is foreign language function and not to do anything weird with the symbols of this function when compiled because this will called from from other languages. This is needed if you plan on doing any FFI. Not doing so means you won’t be able to reference it in other languages.

extern

The extern keyword is used in two places in Rust. One is in conjunction with the crate keyword to make your Rust code aware of other Rust crates in your project, i.e. extern crate lazy static. The other use is in foreign function interfaces (FFI).

read more extern

how to create a shared object ie .so file from your Rust program ?

We can create a library in Rust using command below command

We are required to update the Cargo.toml

[dependencies]
[lib]
name = "callc"
crate-type = ["staticlib", "cdylib"]

cdylib is important crate type to specifiy in the Cargo.toml.
--crate-type=cdylib, #[crate_type = "cdylib"] - A dynamic system library will be produced. This is used when compiling a dynamic library to be loaded from another language. This output type will create *.so files on Linux, *.dylib files on macOS, and *.dll files on Windows.

Read more on this here linkage

Cargo new callc


//lib.rs
#[no_mangle]
pub extern fn hello(){
  println!("hello world inside rust ");
}
          

as we discussed earlier , since we use #[no_mangle], we can directly go ahead and compile it.
cargo build --release

How to check the generate .so file ?

A file with extension “.so” will generated in you target->release library.
enter image description here

run the below command to verify the function in the shared object.
$nm -D ‘filename.so’ | grep ‘function_name’
enter image description here

Our external function name is available in the shared object file created.

Create a C program for calling the shared object ( .so file)

create C program to call the hello function in your ‘src’ folder.

hello.c

#include<stdio.h>
#include "hello.h" /*header file to be included

/* the main C program to call the hello function
/* hello function is resides in the .so ( shared object) file

int main(void){
   hello();
}

Below header file needs to be created in your ‘src’ folder

hello.h

/*the header include file should have the function declaration.

void hello();

We should link the .so file while compiling the c program. The below command will link and compile the program.

While compiling the C program, link the .so file the ‘release’ folder.

gcc -o hello hello.c -L. -Isrc /home/naveen/rustprojects/callc/target/release/libcallc.so

Your C program should get compiled and 'hello’e executable should be generated.

How do you call a Rust program from Python.

We are going to use the crate pyo3 for creating python modules.

create the projectfolder using cargo
cargo projectname --lib

enter image description here

Your Cargo.toml file should be updated as below in library and dependency sections.

[lib]
name = "string_sum"
crate_type = ["cdylib"]

[dependencies.pyo3]
 version = "0.7.0"
 features = ["extension-module"]

update the lib.rs with the below code:

use pyo3::prelude::*;
use pyo3::wrap_pyfunction;

#[pyfunction]
///format the 2 numbers as string
fn sum_as_string(x:usize,y:usize) -> PyResult<String>{
    Ok((x+y).to_string())
}

#[pymodule]
fn string_sum(py:Python,m:&PyModule) -> PyResult<()>{
    m.add_wrapped(wrap_pyfunction!(sum_as_string))?;
    Ok(())
}

You can build the Rust program using Cargo build --release

This will create .so object in your cargo release folder. You might need to rename the file libstring_sum.so to string_sum.so

enter image description here

Now you can start import the module to the python interpreter and starting using the function string_sum

You might need to use python3 or 3+ interpreters for pyo3 to work.
As screenshot shows below , you can import the functions defined in the Rust module and use them in python.

enter image description here

Tuesday, August 20, 2019

One of powerful usage of Enum in Rust - Part1

Welcome file

Rust programming - Enum inside Struct

You might have come across this scenario in programming where you want to restrict a field type to restricted custom types.

Especially when you are working with APIs, the some of the field types can be one of the multiple custom types. You don’t want to accept the types other than defined custom types. enum can help you in these scenarios.
,eg event types can be ‘pullevent’, 'pushevent", ‘keypress’,‘mouseevvent’ etc

Example progarm

#[derive(Debug)]
///enum defined for vechicle type field
enum Oneoftwo{
    Car(String),
    Bus(String),
   
}
///struct defined for vehicle.
///Here we are using the defined enum as a type for a field in the struct
struct Vehicle{
    type1:Oneoftwo,
    price:i32,
}
///implementation for the vechile to create new instances
impl Vehicle {
    fn new(type1:Oneoftwo,price:i32) -> Vehicle{
        Vehicle{
            type1,
            price,
        }
    }
}

///main program to demostrate how to use enum inside a struct
fn main() {
    ///creating instances of the struct object
    let honda = Vehicle::new(Oneoftwo::Car("honda".to_string()),4000);
    let tata = Vehicle::new(Oneoftwo::Bus("Tata".to_string()),10000);
     
    println!("{:?},{:?}",honda.type1,honda.price);
    println!("{:?},{:?}",tata.type1,tata.price);
}


output:
Car("honda"),4000
Bus("Tata"),10000

Sunday, August 11, 2019

Learn Python ,RUST and C - blog post 2

Data Types in Python,Rust and C - PART 1

Understanding Data Types in Python,Rust and C progamming languages.

I am not sure whether I can cover all data types comparison in one blog post. So I might be doing it in multiple blog posts.
Standard Data Types:*
  • Numbers.
  • String.
  • List.
  • Tuple.
  • Dictionary.
Numbers Datatypes
**Python programming **
Type format description
int a = 10 signed integer
long 356L Long integers
float a = 12.34 floating point values
complex a = 3.14J (J) Contains integer in the range 0 to 255
enter image description here
The variable types are dynamically typed in python.So when you assign a value to a variable, python compiler determines the type of the variable.
But once the type is defined , the defined operations can only be performed for that particular type.
but you can change the type of the variable by assigning a different value. So in python it is the developer responsibility to maintain and make sure type of the variable intact through out the program.
For example,
>a = 10;
>a = "naveen" //allowed
>a = a + 1 // not allowed
>a = 11  //allowed
>a = a +1 // now this is allowed
**Rust programming **:
Length signed unsigned
8bit i8 u8
16bit i16 u16
32bit i32 u32
64bit i64 u64
128 bit i128 u128
arch isize usize
signed and unsigned means whether a number can be -ve or +ve.
Each signed variant can store numbers from -(2n -1) to 2n-1
unsigned variants can vary from 0 to 2n-1
eg: u8 means 0 to 28-1 = 128
signed means -28-1 to 28-1
Rust’s floating-point types are f32 and f64, which are 32 bits and 64 bits in size, respectively.
In the below example we assigned a value 129 to varialble ‘b’ which is of ‘i8’ ,so the program won’t get compiled.
fn main(){
   let a:i8 = 127;
   let b:i8= 129;  //this will cause overflow.
   let c:f32=123.32;
   let d:i8 = -127;
   let e:u8 = 127;

  println!("{}.{},{},{},{}",a,d,c,e,b);
}

when you compile the program,

error: literal out of range for `i8`
 --> datatype.rs:3:13
  |
3 |   let b:i8= 129;
  |             ^^^
  |
  = note: #[deny(overflowing_literals)] on by default


String:
Strings in python can be denoted by single quotes(’), double quotes("), triple quotes(""").
name1    = 'naveen'
name2    = "davis"
fullname = """naveen davis
                       vallooran"""
python3
It possible to access the characters in a string using their index in python.
python-string2
nam1[0] prints the character 'n'

Saturday, August 10, 2019

Learn Python ,Rust and C - blog post 1

Learn Python ,Rust and C - session 1

Learning Python ,Rust , C programming together

I have seen in different forums that people ask following questions " Why should I learn multiple programming language ?" or “Should I learn C or Python or JavaScript ?”.
My opinion would be , you can concentrate on one language and learn it deep. But as a developer some point of time it is important that to learn multiple programming languages and understand the difference and know where these languages are strong. This would helps you take decision on which language to use for your application.

Python Hello world : you can run this program from a python interpreter or save the program in in file and run with python.
install python - python

//python programming 
print("hello world")

enter image description here

Rust Hello world
The below Rust program needs to be saved in filename.rs and compiled with Rust compiler.t
Please follow the link for Rust installation How to install rust

///Rust programming 
fn main(){
 println!("hello world");
}

enter image description here

or You can create a rust programs using Cargo.
The link will help you to install cargo - Cargo
enter image description here

C Hello world
You preferred to have GCC compiler to generate C program executable.
installation -
Ubuntu GCC installation
Windows GCC

The program can be saved in a helloworld.c and compile and execute as below .

enter image description here

#include<stdio.h>
int main(){
    printf("hello world\n");
}

Python Rust C
Interpreting language compiled language compiled language
platform independent cross platform
individual building or compilation for each platform that it supports,
platform dependent
Strongly typed
dynamically typed
strongly typed and dynamically typed weekly typed and statically typed
objected oriented Object-Oriented Programming is a way of modeling programs that originated with Simula in the 1960s and became popular with C++ in the 1990s. There are many competing definitions for what counts as OOP, and under some definitions, Rust is object-oriented; under other definitions, it is not. structural language
inbuilt garbage collector no separate garbage collector,but memory management is done automatically in compile time manual memory management

Tuesday, May 28, 2019

Rust Closure - PART2

Rust Closure - PART2.html

Closure Rust - Cont…

FnMut Trait

This is a continuation of the previous post ( https://naveendavisv.blogspot.com/2019/05/what-is-closures-how-can-it-be-used-in.html) on closures in rust.
Last post we defined the closures as Fn(i32) -> i32 when we want to pass the closure to a function.
Fn trait borrows values from the environment immutably.
But now we will see what is FnMut? .
FnMut can change the environment because it mutably borrows values.
fn double(mut db1:T)
   where T:FnMut() -> i32 {
     println!("{:?}",db1());
  }

fn main() {

  let mut x = 10;
  let  db = || { x*=2; x};
  double(db);
  println!("{:?}",x);
}
The above example, the ‘x’ value will recalculated when we call the closure inside the function ( double)
If we want to change a value in the environment that closure is enclosing, the FnMut trait can be used to define the closure.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5102dff068bc256a59bc9c8c9bf3f92f

Cacher struct with closure.

We will try to build a Cacher Structure as explained in the rust book - https://doc.rust-lang.org/book/ch13-01-closures.html
The need for Cacher structure is explained in the book.
Final part of Cacher structure chapter , the author ask us to create the struct with HaspMap to store the calculated value
I will try to explain the Code wrote in Rust here.
How nice would it be if we can cache the computation intensive function and store it's calculated values in a struct !!
A data structure with a 'Closure' and 'Hashmap' will serve the purpose very nicely.

How do we define Cacher Structure with HashMap ?

We can call data structure as Cacher as it cache/store the result values.
Normally we don’t define the parameter type or return type of a Closure in Rust. But it is must to explicitly declare the field types of a Struct , so we need to explicitly define the 'T' where T is type of the Closure .Rust Compiler needs to understand all the field types of structures in order to allocate memory
resultMap field is Box (A pointer type for heap allocation - https://doc.rust-lang.org/std/boxed/struct.Box.html) reference to Hashmap.
If you don’t know how much memory you are going to use, one option would be “Box” type. Here we defined the field resultMao as Box reference to a HaspMap .
struct Cacher
  where T:Fn(u32) -> u32
  {
    square:T,
    resultMap:Box>,
  }

How do we implement the Cacher Struct ?

We defined the Cacher Struct , the next step would be we need to implement the Struct with methods.
one method that we would require is “new” . This method creates Objects of the Struct. Another one would be “value” - to get the value from the resultMap. As resultMap is reference to a HashMap , we can use “insert” method to add key and value pairs to the hashmap.
Also we can use “get” method to get a value for a key from the haspmap.
impl Cacher
  where T:Fn(u32) -> u32{
  fn new(square:T,mut resultMap: HashMap) -> Cacher{
    Cacher{
              square,
              resultMap:Box::new(resultMap),
           }
  }

  fn value(&mut self,x:u32) -> u32 {


     match self.resultMap.get(&x){
       Some(v) => *v,

       None    => {
                   let v = (self.square)(x);
                      self.resultMap.insert(x,v);
                   v
                  },
    }
  }
}

Rust Playground - https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=d131716d3620b60521a2af2067a32dcd