Saturday, March 14, 2020

How do we make a C program call Rust program

C to Rust.md

Internals of C program call to rust program call

One of the main strength of Rust programming language is that it can easily inter-operate with other programming languages.

But as we know Rust is very very strongly typed language. When you are planning Rust to use some of the C libraries , the C type representation attributes help us .

#[repr( C )]

There are some difference in the type representation for C type corresponding rust type.

We can go-ahead and check what is the size of the below struct
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=7392513792a8d390ce4756a6a8a0ed15

use std::mem;
#[repr(C)]
struct FieldStruct {
    first: u8,
    second: u16,
    third: u8
}

// The size of the first field is 1, so add 1 to the size. Size is 1.
// The alignment of the second field is 2, so add 1 to the size for padding. Size is 2.
// The size of the second field is 2, so add 2 to the size. Size is 4.
// The alignment of the third field is 1, so add 0 to the size for padding. Size is 4.
// The size of the third field is 1, so add 1 to the size. Size is 5.
// Finally, the alignment of the struct is 2 (because the largest alignment amongst its
// fields is 2), so add 1 to the size for padding. Size is 6.
assert_eq!(6, mem::size_of::<FieldStruct>());
}

But if you remove #[repr( c )] , the struct size becomes 4.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f1ad5c57ae7e771ca505058784580a1e

#![allow(unused)]
fn main() {
use std::mem;
struct FieldStruct {
    first: u8,
    second: u16,
    third: u8
}
assert_eq!(6, mem::size_of::<FieldStruct>());
}

How to use a Rust function in C program.

I referred the blog for Sergey Potapov for the details.

In this below program, you would see the function print_hello_from_rust defined with extern keyword and [no_mangle] attribute.
[no_mangle] makes the compiler ignores the unknown symbols and it knows this function is going to get called from other languages.
extern keyword makes the function outside of the our library.

std::ffi::CStr. Representation of a borrowed C string. This type represents a borrowed reference to a nul-terminated array of bytes. It can be constructed safely from a &[ u8 ] slice, or unsafely from a raw *const c_char

In this example we are using *const c_char , but what is c_char ?!

c_char is coming from the standard library ‘os’ module and it is Equivalent to C’s char type.C’s char type is completely unlike Rust’s char type; while Rust’s type represents a unicode scalar value, C’s char type is just an ordinary integer. This type will always be either i8 or u8, as the type is defined as being one byte long

reference : c_char

then why *const in front of it ?

*const are called Raw pointers in Rust.Sometimes, when writing certain kinds of libraries, you’ll need to get around Rust’s safety guarantees for some reason. In this case, you can use raw pointers to implement your library, while exposing a safe interface for your users. Ref: Raw pointers

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=40d05db968f82125fb7660e67710ecff

use std::ffi::{CString,CStr};
use std::os::raw::{c_char,c_int};

#[repr(C)]
pub struct Point{
    x: c_int,
    y: c_int,
}

impl Point {

    fn new(x:c_int,y:c_int) -> Point{
//          println!("Creating a Point with x = {},y = {}",x,y);
          Point {x: x,y: y }

    }
}
#[no_mangle]
pub extern fn create_point(x:c_int,y:c_int) -> *mut Point{

        Box::into_raw(Box::new(Point::new(x,y)))
}
#[no_mangle]
pub extern fn print_hello_from_rust(data: *const c_char ){

    unsafe{
          let c_str =       CStr::from_ptr(data);
          println!("hello from rust {:?}",c_str.to_str().unwrap());
    }
}

You can build the rust program with cargo

cargo new whatland -- lib
cd whatland

edit the lib.rs file with above code.

also make sure that your cargo file has

 [lib]
 name = "whatland"
 crate-type = ["staticlib","cdylib"]

cdylib helps - A dynamic system library creation. This is used when compiling a dynamic library to be loaded from another language. This output type will create *.so files on Linux, *.dylib files on macOS, and *.dll files on Windows.

cargo build --release

C program would need to compile with linking the .so file generated.

I am not detailing the C program compilation, but it detailed in the blogpost: https://www.greyblake.com/blog/2017-08-10-exposing-rust-library-to-c/

You would need to use,

gcc -o ./examples/hello ./examples/hello.c -Isrc  -L. -l:target/release/libwhatlang.so

Passing String parameter from C to Rust function.

We can use the trick of accepting the string as a Raw pointer using *const c_char
In order to print that bytes into a valid string slice in Rust, first we need to convert that to CStr - Representation of a borrowed C string.

then we can convert that to Str ( rust string slice) using c_str.to_str().unwrap()

How to use a C struct in Rust program

I believe , Its always recommended to use #[repr©] when working with C structs, enums because it makes alignment https://doc.rust-lang.org/reference/type-layout.html#the-c-representation

#[repr(C)]
pub struct Point{
   x: c_int,
   y: c_int,
  
  //    x:u8, -- if you are using u8, compilier throws error,
  //    y:u8,    stating "expected `u8`, found `i32'"
  }

We implemented a method new in the rust program so that we can use in the Point struct instance creation.

impl Point {

  fn new(x:c_int,y:c_int) -> Point{

 Point {x: x,y: y }
 }
}

Next in this example we have a export function which will be called from C program to creating Point Struct. For creating any object, we need to memory. In Rust we know that we can allocate heap memory through Box::new.

But we need to return a raw pointer to the C program, which can be done through Box::into_raw which Consumes the Box, returning a wrapped raw pointer.

 23 #[no_mangle]
 24 pub extern fn create_point(x:c_int,y:c_int) -> *mut Point{
 25 
 26         Box::into_raw(Box::new(Point::new(x,y)))
 27 }

C program can now just call the Rust function to create struct.

We need to have our C program included with struct and create function declaration.
header file whatland.h

  1 void print_hello_from_rust();
  2 
  3 typedef struct Point{
  4             int x,y;
  5 }Point;
  6 
  7 Point* create_point(int x, int y);
  

Now we can just call it in our main function in C

 11  Point* p1 = create_point(10,20);
 12     printf("Point={%d},{%d}",p1 -> x,p1 -> y);

Monday, March 9, 2020

Self learn to write a File read program in Rust

cargo-test_debugging

Write a program to read a File in Rust

When I got this question first time, I don’t know where to start with this in Rust.
If this question was to write the same prgoram in C :

C is a school taught language and we know that C 's stdio has fopen, fclose ,fgets etc … because of that I wouldn’t have worried what/how fgets read or type conversion challenges are handled.

Here in Rust , I don’t which libraries/modules needs to imported
libraries are called crates in rust.
Some of the questions came to mind where

  1. what are all the standard libraries needed for this ?
  2. is there any method called ‘read’ in Rust ?
  3. how to get the file descriptor , is there any File Open ?

Looks like all these details are well documented in
std lib in rust

Finally I just started writing the program with whatever read method that I found somewhat relevant

  1 use std::fs::File;
  2 
  3 fn main() {
  4 
  5     let f = File::open("a.txt");
  6 
  7     match f.read() {
  8           Ok(x) => { println!("file contents = {}",x) },
  9           Err(e) => { println!("Error") },
 10     }
 11 }

Oops Error…
compiler complaints …

 error[E0599]: no method named `read` found for enum `std::result::Result<std::fs::File, std::io::Error>` in the current scope
 --> src/main.rs:7:13
  |
7 |     match f.read() {
  |             ^^^^ method not found in `std::result::Result<std::fs::File, std::io::Error>`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0599`.
error: could not compile `std_learning1`.

Ok. read method is not there in fs inmodule.
We will goahead search where is the read in rust lang documentation.

I used std::fs::read , but if we read through the documentation carefully , we will understand that this is for small files and that can be parsed to a string format type like SocketAddr.

enter image description here

9   |     match std::fs::read(f) {
    |                         ^ the trait `std::convert::AsRef<std::path::Path>` is not implemented for `std::result::Result<std::fs::File, std::io::Error>

This error actually gives a clue that our “f” file object is “std::result::Result<std::fs::File, std::io::Error>”

I know that I can handles the error types with just adding “?” to file open.

But still we haven’t got which read method to use . Again going to back to documentation and search - I found a read that reasonable choice which is std::io:: Read::read

enter image description here

(Note:wrongly highlighted above)

just brought std::io::Read trait alone to the program for now.

as the function signature says -
fn read(&mut self, buf: &mut [u8]) -> Result<usize >

 1 use std::fs::File;
 2 use std::io::Read;
 4 
 5 fn main()  {
 6 
 7     let mut f = File::open("a.txt")?;
 8     let mut buf = [0;30];
 9     let n =  f.read(&mut buf[..]);
 
14 
15     println!("{:?}",&buf[0..n]);
16 
18 }


mainly two errors

  | |
7  | |     let mut f = File::open("a.txt")?;
   | |                 ^^^^^^^^^^^^^^^^^^^^ cannot use the `?` operator in a function that returns `()`
8  | |     let mut buf = [0;30];
error[E0308]: mismatched types
  --> src/main.rs:15:29
   |
15 |     println!("{:?}",&buf[0..n]);
   |                             ^ expected integer, found enum `std::result::Result`
   |
   = note: expected type `{integer}`
              found enum `std::result::Result<usize, std::io::Error>`

error: aborting due to 2 previous errors

we need to give return type in the main() function as we used “?” in line 7 and also we need to return Ok(()) in the main

We need to add io::Result<()> as return type in the main function, so for that we would need to include std::io;

So code would looks something like below:

  1 use std::fs::File;
  2 use std::io::Read;
  3 
  4 use std::io;
  5 //use std::io::prelude::*;
  6 
  7 fn main() -> io::Result<()>  {
  8 
  9     let mut f = File::open("a.txt")?;
 10     let mut buf = [0;30];
 11     let n =  f.read(&mut buf[..])?;
  17     println!("{:?}",&buf[0..n]);
 18 
 19    Ok(())
 20 }



But the output is still bytes.

   Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `/home/naveen/rustprojects/mar2020/std_learning1/target/debug/std_learning1`
[84, 104, 105, 115, 32, 105, 115, 32, 102, 105, 114, 115, 116, 32, 82, 117, 115, 116, 32, 115, 116, 100, 32, 108, 105, 98, 114, 97, 114, 121]

We can convert the bytes to string type using below statements.


 println!("{:?}",String::from_utf8((&buf[0..n]).to_vec()).unwrap());

One you might have noticed is that we need to handle the buffer explicitly.

So let us look again for some other read method available.

The one I found interesting is BufRead Trait which is a type reader that handle internal buffer.

Let us search for what are all the methods implementing this.

enter image description here

But how it can be used or related to File struct.

We know that File implements Read trait.

enter image description here

BufReader implements Read
BufReader implements BufRead as well.
So we can go-ahead are use/create an instance of BufReader on a File object.

Once we convert File object to an instance of BufReader , we can use methods like
lines
read_lines
read_until
split

  1 use std::fs::File;
  2 //use std::io::Read;
  3 
  4 use std::io::{self,BufReader};
  5 
  6 //include io  prelude which import all important structs and implementation
  7 //in this case it import all supporting BufReader implementation ( eg :  //BufRead)  for File struct.
  9 
 10 
 11 use std::io::prelude::*;
 12 
 13 fn main() -> io::Result<()>  {
 14 
 15     let mut f = File::open("a.txt")?;
 16 //    let mut buf = [0;30];
 17 
 18     let buf = BufReader::new(f);
 19 
 20  //   let n =  f.read(&mut buf[..])?;
 21 
 22    for line in buf.lines(){
 23         println!("{:?}",line);
 24    }
 25 
 26 
 27    Ok(())
 28 }



Diagram might be not fully correct. But I am trying to picturize the File read program modules and important internals.

Some notes in the above snippet:

  1. include io::prelude which import all important structs and implementation. in this case it import all supporting BufReader implementation ( eg : BufRead) for File struct.

  2. commented some of the lines(not deleted) which we used prior version of the program to understand the difference.

Output :

Ok("This is first Rust std library learning program.")

if we do an unwrap(), we will get the line string itself.

Conclusion:
Hope this helps you to understand how different traits are connected together at least for Rust File read program and which std modules needs to be imported for it. This same module/trait analysis required when you are working with external crates. It is important reading through the crate documentation and understand which traits are matching your requirement and which are needs to be imported and ready to use for their methods.

Saturday, March 7, 2020

regex programs should be a must tool for a programmer

Rust Regex:

I haven't given much attention to pattern matching. but I think it should be must taught topic for every programming student. having said that,  we can get into the my example



we can get into the Rust regex crate https://crates.io/crates/regex
and https://docs.rs/regex/1.3.4/src/regex/lib.rs.html#1-785





Since Rust handles string pretty nicely, regex expression matching is quite easy even if for my native language - Malayalam

you can either create an Rust regex object with Regex::new(r"(മല)") or RegexSetBuilder::new(&[r"ജാവ"]).case_insensitive(true) .build()?;


if you really want to check each malayalam character , then you can use Regex::new(r"(p\{Malayalam}).unwrap()

For more detail how to use regex : 

https://www.regular-expressions.info/brackets.html

https://medium.com/factory-mind/regex-tutorial-a-simple-cheatsheet-by-examples-649dc1c3f285


Code details: regex malayalam search .