Taking O’Reilly’s “Rust Crash Course”

I spend some “off-time” to burn some more Rust into my brain, this time with a quick, online class

My “classroom” was more “virtual” than this!!

I am off a week from work to recharge and rid myself of the creeping “burned-out” feeling. However, that doesn’t mean (for me) that I avoid my computer! Instead, I’ll spend some of the free time to attend a 4-hour online class offered through O’Reilly. The class is “Rust Crash Course: Learn Enough to Get Over the Hump” given by Nathan Stocks. There is another session coming up in September, 2019 if this might work well for you. I’ve been through both of the primary Rust books I have listed in the sidebar on the right, so I might be further along than some of the students. However, I’m really trying to drill some Rust into my head, so I like the idea of multiple (and varied) passes over the basics.

Attention, class!

My Uplift standing desk and workstation setup for working from home
Ah, more like it – this is my virtual classroom 🙂

The class has a GitHub repository as a resource, which Stocks takes us through as the class moves along during the 4 hours. Sure enough, he stays fairly “basic” and reading the Rust books ahead of time is not necessary – though previous programming experience IS needed. On the other hand, I have not found a better instruction guide for Rust specifically, than the official Rust book, The Rust Programming Language, which is available to read online for free. Different people learn differently though, and this class was a nice reinforcement. Also, reading those two books before taking this class gave me great background information about the topics Stocks breezed through.




The instructor does a great job of quickly tackling the questions at the end of each section during the class. This is a good introductory Rust crash course – he takes us through most idiomatic Rust concepts and moves pretty fast through the material. Think of this as a shallow dive, covering the “first couple of pages” from each of the basic Rust-book chapters. He helpfully includes quick, on-screen Rust code editing along the way, plus takes questions after covering each section.

More often than not, I’ve experienced streaming trouble during an O’Reilly class in the past, and this was no exception. About half-way through, we lost the instructor completely. O’Reilly staff was right there, though, to work the problem and get Stocks back to us. Other times, I got stuck “buffering…” while trying to follow his live video. During those instances, the instructor’s bandwidth graph was “red” instead of “green”. That was just one clue that made me believe it wasn’t me. I think the issue here is that the Internet infrastructure in the United States is, unfortunately, embarrassingly lackluster compared to many other countries.

Free “Trial” it Out!

I do recommend you signup for an O’Reilly account. Not only do you get access to their published books to read online, but there are also many online classes you can take as well! See if your company will cover the cost, or if not, $399 for a year’s access is less than most would pay for a local college course and you can squeeze much more use out of this over the year! At least try the 10-day free trial to see what’s available! No affiliate links here, just passing on the information!

Rust Functions, Modules, Packages, Crates, and You

Wooden pallets stacked one on top another
I know the code is in here… somewhere.

Come to find out, I’m learning Rust from old documentation. Both of the printed Rust books I have are for the pre-“2018 edition” and I think that’s contributing to some confusion I have about functions, modules, packages, and crates. A new version of the official book is coming out in the next month or so – I have a link to it through Amazon in the right sidebar. If you’ve been reading the online documentation, you’re ok – it is updated fir the “2018-edition”. I’ve looked at some of these parts of Rust before, but I recently found another new resource, the Edition Guide, which clears up some of my issues. Especially of interest here, is the section on Path Clarity which heavily influenced by RFC 2126 that improved this part of Rust.

I learned some of the history (and excitement) of RFC 2126 while listening to the Request for Explanation podcast, episode 10. Anyway, let’s go back to basics and have a look at Rust functions, modules, packages and crates as the language sits in mid-2019. I’ll present some examples from my web application we’ve been looking at. I’m going to cut out unnecessary bits to simplify things, so a “…” means there was more there in order for this to compile. You can always see whatever state it happens to be in, here.

Crates and Packages

A Rust crate (like Rocket or Diesel) is a binary or library of compiled code. A binary crate is runnable while a library crate is used for its functionality by being linked with another binary. A package (like my web app) ties together one or more crates with a single Cargo.toml file. The toml file configures the package‘s dependencies and some minimal information about compiling the source. A binary crate will have a src/main.rs with a main() function which directs how the binary runs. A library crate will have a src/lib.rs which is the top layer of the library. This top layer directs which pieces inside are available to users of the library.

Rust Functions

Functions are easy – subroutines in your source code. A function starts with fn, possibly receives some parameters and might return a value. Also, a function may be scoped as public or kept private. The main() function inside src/main.rs is a special function that runs when the binary is called from the command line. It dictates the start of your program and you take control from there. You may create other functions, just avoid reserved words (or use the r# prefix to indicate you mean YOUR function, not the reserved word, for instance r#expect if you want to name a function “expect”). Very similar to functions, are methods and traits, which we’ve looked at before.

<src/lib.rs>

...
use diesel::prelude::*;
...
pub fn setup_db() -> PgConnection {
    PgConnection::establish(&CONFIG.database_url)
        .expect(&format!("Error connecting to db"))
}

setup_db() is a fairly simple function – it accepts no incoming parameters and returns a database connection struct called PgConnection. It has pub before fn to indicate it is a “public” function. Without that, my web application bin/src/pps.rs could not call this function – it would not be in scope. Without pub, setup_db() would only be callable from within src/lib.rs. Since I am designing my application as a library crate, I choose to put setup_db() in the main src/lib.rs file. My binary that I will use to “run” my web application is in src/bin/pps.rs and contains a main() function.

Let’s look at the return type, PgConnection. This is a struct defined by the database ORM library crate, Diesel. The only way I could write a function that returns this particular type of struct is because I have use diesel::prelude::*; at the top (and it’s in the toml file as well). The Diesel library crate provides prelude as a simple way to bring in all Diesel has to offer my package. Diesel provides the PgConnection struct as public (or what good would the crate be), so I can now use that struct in my code. This also gives me the (method or trait, how can you tell?) establish(). Just like you’d call String::new() for a new string, I’m calling PgConnection::establish() for a new database connection and then returning it (see, no trailing ; on the line).




Rust Modules

Functions (and other things) can be grouped together into a Module. For instance, setup_logging() is also in src/lib.rs. However, I could have wrapped it inside a named module, like so:

<src/lib.rs>

...
pub mod setting_up {
    ...
    use logging::LOGGING;
    use settings::CONFIG;

    pub fn setup_logging() {
        let applogger = &LOGGING.logger;

        let run_level = &CONFIG.server.run_level;
        warn!(applogger, "Service starting"; "run_level" => run_level);
    }
}

Now it is part of my setting_up module. Here also, the module needs to be pub so that my application can use it and the public functions inside it. Now all of the enums and structs and functions inside the module setting_up are contained together. As long as they are public, I can still get to them in my application.

Notice I use logging::LOGGING; and use settings::CONFIG; These bring in those two structs so I can use the global statics that are built when then the application starts. I included pub mod logging; and pub mod settings; at the top level, in src/lib.rs, so they are available anyplace deeper in my app. I just need to use them since I reference them in this module’s code.

Splitting firewood with an axe

Split, for Clarity

On the other hand, instead of defining a module, or multiple modules, inside a single file like above, you can use a different file to signify a module. This helps split out and separate your code, making it easier to take in a bit at a time. I did that here, with logging.rs:

<src/logging.rs>

...
use slog::{FnValue, *};

pub struct Logging {
    pub logger: slog::Logger,
}

pub static LOGGING: Lazy<Logging> = Lazy::new(|| {
    let logconfig = &CONFIG.logconfig;

    let logfile = &logconfig.applog_path;
    let file = OpenOptions::new()
        .create(true)
        .write(true)
        .truncate(true)
        .open(logfile)
        .unwrap();

    let applogger = slog::Logger::root(
        Mutex::new(slog_bunyan::default(file)).fuse(),
        o!("location" => FnValue(move |info| {
        format!("{}:{} {}", info.file(), info.line(), info.module(), )
                })
        ),
    );

    Logging { logger: applogger }
});

I have a struct and a static instance of it, both of them public, defined in logging.rs. logging.rs becomes a module of my library crate when I specify it. At the top of src/lib.rs I have pub mod logging; which indicates my library crate uses that module file logging.rs and “exports” what it gets from that module as public (so my bin/src/pps.rs application can use what it provides).

In this case, you also see I use slog::{FnValue, *}}; which is like use slog::FnValue; (which I need for the FnValue struct) and use slog::*; which gives me the fuse struct and the o! macro. I was able to combine those into a single use statement to get just what I needed from that external crate.

The old books I have been referencing have you declaring the third-party crates you want to use in your application in your Cargo.toml file (which is still required), but also you’d have to bring each one in with an extern crate each_crate; at the top of main.rs or lib.rs. Thankfully, that’s no longer needed… 99% of the time. In fact, I had a long list of those myself – I am surprised cargo build didn’t warn me it was unneeded. Actually, I do have one crate I am using which still needs this “2015-edition” requirement: Diesel. Apparently, it is doing some fancy macro work and/or hasn’t been upgraded (yet?) for the “2018-edition” of Rust, so at the top of src/lib.rs, I have:

#[macro_use]
extern crate diesel;

A Few Standards and TOMLs

The Rust crate std is the standard library, and is included automatically. The primitive data types and a healthy list of macros and keywords are all included. But, if you need filesystem tools: use std::fs; and if you need a HashMap variable, you’ll need to use std::collections::HashMap; And yes, all external crates you depend on inside your source will need to be listed in Cargo.toml. This configuration helps you though – it updates crates automatically as minor versions become available, but does NOT update if a major version is released. You will need to do that manually, so you can test to see if the major release broke anything you depended on in your code. Here is a piece of my ever-growing Cargo.toml file for the web application so far:

...
[dependencies]
slog = "2.5.0"
slog-bunyan = "2.1.0"
base64 = "0.10.1"
rand = "0.7.0"
rand_core = "0.5.0"
rust-crypto = "0.2.36"
config = "0.9.3"
serde = "1.0.94"
serde_derive = "1.0.94"
serde_json = "1.0.40"
once_cell = "0.2.2"
dotenv = "0.14.1"
chrono = "0.4.7"
rocket = "0.4.2"
rocket-slog = "0.4.0"

[dependencies.diesel]
version = "1.4.2"
features = ["postgres","chrono"]

[dependencies.rocket_contrib]
version = "0.4.2"
default-features = false
features = ["serve","handlebars_templates","helmet","json"]

Better Logging for the Web Application

I replace log and simple_logger with slog and sloggers, and then change again

Part of a Series: Designing a Full-Featured WebApp with Rust
Part 1: Piecing Together a Rust Web Application
Part 2: My Next Step in Rust Web Application Dev
Part 3: It’s Not a Web Application Without a Database
Part 4: Better Logging for the Web Application
Part 5: Rust Web App Session Management with AWS
Part 6: OAuth Requests, APIs, Diesel, and Sessions
Part 7: Scraping off the Dust: Redeploy of my Rust web app
Part 8: Giving My App Secrets to the AWS SecretManager

A journal ledger of accounts... like logging but only uses numbers
See… structured logging…

When I last left my sample web application, I was doing simple logging to the terminal which is not useful for very long. I want structured logging and I want the app and Rocket to write to (separate) files. So, let’s switch out the crates log and simple_log for slog and sloggers to get better logging for my web application.

Application Logging

Slog‘s “ambition is to be The Logging Library for Rust” and it sure seems adaptable. There are several “helper” crates to log to the terminal or syslog, log as json, etc. However, slog is complex! Which seems to be the main reason sloggers came about. Sloggers brings you the most useful features of slog without the complex setup. It happens to include an example of configuring via an inline TOML string. I’ve already got my CONFIG global pulling from a TOML file, so I add this to conf/development.toml:

<conf/development.toml>

...
[logconfig]
type = "file"
format = "compact"
source_location = "module_and_line"
timezone = "local"
level = "debug"
path = "log/error.log"
rotate_size = 1073741824
rotate_keep = 2

[webservice]
weblog_path = "log/access.log"
...

Plus, I add logging.rs to set up the global LOGGING instance so everyone can log; just like everyone can pull from the global CONFIG instance:

<logging.rs>

use crate::settings::CONFIG;
use crate::sloggers::{Build, Config};
use once_cell::sync::Lazy;

#[derive(Debug)]
pub struct Logging {
    pub logger: slog::Logger,
}

pub static LOGGING: Lazy<Logging> = Lazy::new(|| {
    let logconfig = &CONFIG.logconfig;

    let builder = logconfig.try_to_builder().unwrap();
    let logger = builder.build().unwrap();

    Logging { logger: logger }
});

Sloggers is really easy to use to set up a slog instance. Here, I simply pull the logconfig from my global CONFIG, build the logger, and store it in my new OnceCell Lazy LOGGING global.




And then, Web Access Logging

A spider web, wet from rain...

At this point, application logging is going to my new log/error.log file, but Rocket logging is still coming to the screen. This is actually good – as I mentioned, I want to have them going to separate files anyway. So now with some slogging experience, it’s a simple matter to set up a second file, specifically for Rocket. I do need to add yet another crate, this time rocket_slog – so that Rocket internals can glom on to the slog instance that I make. Here is the new start_webservice function:

<src/lib.src>

...
pub fn start_webservice() {
    let logger = &LOGGING.logger;

    let weblog_path = &CONFIG.webservice.weblog_path;
    let bind_address = &CONFIG.webservice.bind_address;
    let bind_port = &CONFIG.webservice.bind_port;

    // start rocket webservice
    let version = include_str!("version.txt");

    let mut builder = FileLoggerBuilder::new(weblog_path);
    builder.level(Severity::Debug);
    let weblogger = builder.build().unwrap();
    let fairing = SlogFairing::new(weblogger);

    warn!(
        logger,
        "Waiting for connections..."; "address" => bind_address, "port" => bind_port, "version" => version);

    rocket::ignite()
        .attach(fairing)
        .attach(Template::fairing())
        .attach(SpaceHelmet::default())
        .mount("/", routes![routes::index, routes::favicon])
        .mount("/img", StaticFiles::from("src/view/static/img"))
        .mount("/css", StaticFiles::from("src/view/static/css"))
        .mount("/js", StaticFiles::from("src/view/static/js"))
        .launch();
}

Now I have log/error.log for application logging (I’ll rename that later to log/app.log) and log/access.log for Rocket logging. But I still don’t have structured logs!

Finally, Structured Logs

I try several ways (with my limited Rust knowledge and experience) but it appears that with sloggers ease of use, you trade away options if you want to get fancy. At least, I couldn’t figure out how to get slog_json or something called slog_bunyan to work with sloggers. It looks like I have to deal with slog directly.

Later, on another night, I tackle these changes. I dump sloggers and, though much searching around, I end up with a merge of two examples I find (from slog and slog-bunyan). Thankfully, I am getting a little better at understanding the API docs that are posted for crates! Anyway, here is how the app logging setup looks now:

<src/logging.rs>

...
pub static LOGGING: Lazy<Logging> = Lazy::new(|| {
    let logconfig = &CONFIG.logconfig;

    let logfile = &logconfig.applog_path;
    let file = OpenOptions::new()
        .create(true)
        .write(true)
        .truncate(true)
        .open(logfile)
        .unwrap();

    let applogger = slog::Logger::root(
        Mutex::new(slog_bunyan::default(file)).fuse(),
        o!("location" => FnValue(move |info| {
        format!("{}:{} {}", info.file(), info.line(), info.module(), )
                })
        ),
    );

    Logging { logger: applogger }
});

Notice I still pull the applog_path from the CONFIG and then create that filehandle. Next, a single but complicated call into slog returns a Logger instance, which I store into my global. The o! macro, provided by the slog crate, lets me add a JSON field to every log record that gets written. I copy from an example and add the file, line number, and module name where the log was generated. I’ll probably come back to this later and add more data. Also, notice I am using slog_bunyan for the JSON formatting. slog-json actually recommends using slog_bunyan for a “more complete output format.” Bunyan logging seems to originate as a node.js module. Lastly, I also change the weblogger in the same way – now both are JSON logs and still log to separate files.

This actually wasn’t as complicated as I feared. I’ve lost some of the easily configurable sloggers features like auto file rotation, but I’ll figure that out as well – it’s probably actually just a slog feature that I need to enable.

Results, JSON Style

Here is how both look now – I think next post I’ll work on a JSON log viewer so we can watch these easier, as we develop.

<log/app.log>

{"msg":"Service starting","v":0,"name":"slog-rs","level":40,"time":"2019-07-20T03:40:13.781382592+00:00","hostname":"ip-169-254-169-254","pid":6368,"location":"src/lib.rs:48 ppslib","run_level":"development"}
{"msg":"Waiting for connections...","v":0,"name":"slog-rs","level":40,"time":"2019-07-20T03:40:13.784899718+00:00","hostname":"ip-169-254-169-254","pid":6368,"location":"src/lib.rs:76 ppslib","version":"0.1.0","port":3000,"address":"0.0.0.0"}

<log/access.log>
{"msg":"listening","v":0,"name":"slog-rs","level":30,"time":"2019-07-20T05:31:21.068959173+00:00","hostname":"ip-169-254-169-254","pid":7700,"address":"http://0.0.0.0:3000"}
{"msg":"request","v":0,"name":"slog-rs","level":30,"time":"2019-07-20T05:31:22.402815352+00:00","hostname":"ip-169-254-169-254","pid":7700,"uri":"\"/\"","method":"Get"}
{"msg":"response","v":0,"name":"slog-rs","level":30,"time":"2019-07-20T05:31:22.404425952+00:00","hostname":"ip-169-254-169-254","pid":7700,"status":"200 OK","route":"\u001b[32mGET\u001b[0m \u001b[34m/\u001b[0m \u001b[36m(\u001b[0m\u001b[35mindex\u001b[0m\u001b[36m)\u001b[0m"}

Again, if all that was complicated to follow and you prefer to look at the git commits as I went through this ordeal, check out the repository.