Rust Web App Session Management with AWS

Even more libraries added to my web app as I introduce Amazon’s DynamoDB to handle my session storage

Part of a Series: Designing a Full-Featured WebApp with Rust
Part 1: Piecing Together a Rust Web Application
Part 2: My Next Step in Rust Web Application Dev
Part 3: It’s Not a Web Application Without a Database
Part 4: Better Logging for the Web Application
Part 5: Rust Web App Session Management with AWS
Part 6: OAuth Requests, APIs, Diesel, and Sessions
Part 7: Scraping off the Dust: Redeploy of my Rust web app
Part 8: Giving My App Secrets to the AWS SecretManager

Printed security pass - an old school session management practice
Didn’t someone mention a cookie too!? Where’s my cookie!?

I’ve worked on some more bits with my pretend web application I introduced a few posts ago. Thinking about the technology needed to let users register and later log in, session management seems like a good first step to take. I would like to store the sessions in Amazon’s DynamoDB too, just for fun and experience. I’ve pushed several commits to the repository lately, so let me go through some of the changes enabling Rust web app session management with AWS DynamoDB.

First, I searched and read up a bit on best practices for session management. I found a Session Management Cheat Sheet from OWASP. Also, more generically, 12 Best Practices for User Account, Authorization, and Password Management on the Google Cloud Blog. I haven’t addressed many of these yet, but I’ll come back to resources like this several times in the future. I was trying to find language-independent whitepapers or guides on web app session management… and something not a decade old. If anyone has a definitive resource for this, please share!

Rusoto to the Rescue

First, let’s get our web application interfacing with AWS. Rusoto is a big set of crates for using AWS services. Rusoto Core (of course) and DynamoDB are all I need for now, but I am sure I’ll come back for more! To get set up, I created a sessions table in my AWS console and created a programmatic IAM role called pinpointshooting which has full access to the table. When creating a programmatic role, you are given credential access keys which I put into my .env file at the root of the pinpointshooting project. First, connecting to my AWS account with those access keys is a simple matter of:

DynamoDbClient::new(Region::UsEast1)

I had to think about the process I would use to store the session id in a cookie, and when/how it gets created, verified, updated, and deleted. I came up with a simple sketch to make it more clear in my mind. Note that I mention a cache both in my sketch and in some of the comments, but I haven’t done anything about that yet.

  • When a user arrives with no cookie – we need to create a session with some appropriate defaults, write it to the db, and send the cookie in the resulting response.
  • If the user arrives with a cookie – we need to pull the session from the DB (if it really exists) and do some verification and expiration checking. If everything looks good, we pull the session details into our struct. Otherwise, we delete that invalid session data and create a new session like above.
  • When the user selects to log out – we again check to make sure the session is valid, but then delete the session, delete the cookie, and log the user out.



Here are those steps above, as I have them in code at this point!

Step 1: Search for or Create the Session

// Check for sessid cookie and verify session or create new
// session to use - either way, return the session struct
pub fn get_or_setup_session(cookies: &mut Cookies) -> Session {
    let applogger = &LOGGING.logger;
    let dynamodb = connect_dynamodb();

    // if we can pull sessid from a cookie and validate it,
    // pull session from cache or from storage and return
    if let Some(cookie) = cookies.get_private("sessid") {
        debug!(applogger, "Cookie found, verifying";
            "sessid" => cookie.value());

        // verify from dynamodb, update session with last-access if good
        if let Some(mut session) = verify_session_in_ddb(&dynamodb, &cookie.value().to_string()) {
            save_session_to_ddb(&dynamodb, &mut session);
            return session;
        }
    }

    // otherwise, start a new, empty session to use for this user
    let mut hasher = Sha256::new();
    let randstr: String = thread_rng()
        .sample_iter(&Alphanumeric)
        .take(256)
        .collect();
    hasher.input(randstr);
    let sessid = format!("{:x}", hasher.result());

    cookies.add_private(Cookie::new("sessid", sessid.clone()));

    let mut session = Session {
        sessid: sessid.clone(),
        ..Default::default()
    };

    save_session_to_ddb(&dynamodb, &mut session);
    session
}

This will check the (private) cookie sessid, if it came along with the http request, and try to fetch it from DynamoDB and then verify it (see below). If that succeeds, we update the session (with an updated last-access timestamp) and return the session struct. Otherwise, we create a new session id, with some appropriate defaults for now, by getting a hash of a random string of characters. Hrm, I should make sure I didn’t just happen to intersect with an existing sessid at this point! Anyway, we then just add (or update) the sessid cookie to be returned with the http response. Since we are using the private cookie feature of Rocket, the value is actually encrypted. The user can’t see what their real session id is or try to manufacture one. Lastly, if we just created a new session, we save it to the session table and return it.

Step 2: Verify a Session

// Search for sessid in dynamodb and verify session if found
// including to see if it has expired
fn verify_session_in_ddb(dynamodb: &DynamoDbClient, sessid: &String) -> Option<Session> {
    let applogger = &LOGGING.logger;

    let av = AttributeValue {
        s: Some(sessid.clone()),
        ..Default::default()
    };

    let mut key = HashMap::new();
    key.insert("sessid".to_string(), av);

    let get_item_input = GetItemInput {
        table_name: "session".to_string(),
        key: key,
        ..Default::default()
    };

    match dynamodb.get_item(get_item_input).sync() {
        Ok(item_output) => match item_output.item {
            Some(item) => match item.get("session") {
                Some(session) => match &session.s {
                    Some(string) => {
                        let session: Session = serde_json::from_str(&string).unwrap();
                        match session.last_access {
                            Some(last) => {
                                if last > Utc::now() - Duration::minutes(CONFIG.sessions.expire) {
                                    Some(session)
                                } else {
                                    debug!(applogger, "Session expired"; "sessid" => sessid);
                                    delete_session_in_ddb(dynamodb, sessid);
                                    None
                                }
                            }
                            None => {
                                debug!(applogger, "'last_access' is blank for stored session"; "sessid" => sessid);
                                delete_session_in_ddb(dynamodb, sessid);
                                None
                            }
                        }
                    }
                    None => {
                        debug!(applogger, "'session' attribute is empty for stored session"; "sessid" => sessid);
                        delete_session_in_ddb(dynamodb, sessid);
                        None
                    }
                },
                None => {
                    debug!(applogger, "No 'session' attribute found for stored session"; "sessid" => sessid);
                    delete_session_in_ddb(dynamodb, sessid);
                    None
                }
            },
            None => {
                debug!(applogger, "Session not found in dynamodb"; "sessid" => sessid);
                None
            }
        },
        Err(e) => {
            crit!(applogger, "Error in dynamodb"; "err" => e.to_string());
            panic!("Error in dynamodb: {}", e.to_string());
        }
    }
}

That’s some deep nesting – I’m still a newbie at Rust coding, so there is probably a better way to write this. If I didn’t care about logging the problems, I could probably shorten this by using the “?” operator at the end of each step. Maybe I’ll change that eventually, but for now I need to debugging logs to see that things are working.

This chain tries to retrieve the session attribute associated with the sessid (session id encoded in the cookie) and for now, just makes sure it isn’t too old. If that simple check is ok, we deserialize and return that Some(session struct). In all other cases, we return None and a new session will be generated. For cases where a session was present, but determined to be invalid or expired, we also delete the session from DynamoDB.

Step 3: Save (or Update) the Session

// Write current session to dynamodb, update last-access date/time too
fn save_session_to_ddb(dynamodb: &DynamoDbClient, session: &mut Session) {
    let applogger = &LOGGING.logger;

    session.last_access = Some(Utc::now());

    let sessid_av = AttributeValue {
        s: Some(session.sessid.clone()),
        ..Default::default()
    };
    let session_av = AttributeValue {
        s: Some(serde_json::to_string(&session).unwrap()),
        ..Default::default()
    };
    let mut item = HashMap::new();
    item.insert("sessid".to_string(), sessid_av);
    item.insert("session".to_string(), session_av);

    let put_item_input = PutItemInput {
        table_name: "session".to_string(),
        item: item,
        ..Default::default()
    };

    match dynamodb.put_item(put_item_input).sync() {
        Ok(_) => {}
        Err(e) => {
            crit!(applogger, "Error in dynamodb"; "err" => e.to_string());
            panic!("Error in dynamodb: {}", e.to_string());
        }
    };
}

Here, we simply take the session data we are passed, update the last_access to now() and then write the serialized session struct to DynamoDB. Also note, I’m storing the session id inside the session data itself. I’m not sure yet if this is handy duplication, a security concern, or irrelevant.

Step 4: Clean-up After Ourselves

Repairman cleaning up his work, but did anyone check his security badge!?
Hey! Are you a security concern??

Just for completeness, here is the function to drop a session from DynamoDB once we have determined it is invalid (expired).

// Delete session from dynamodb
fn delete_session_in_ddb(dynamodb: &DynamoDbClient, sessid: &String) {
    let applogger = &LOGGING.logger;

    let av = AttributeValue {
        s: Some(sessid.clone()),
        ..Default::default()
    };

    let mut key = HashMap::new();
    key.insert("sessid".to_string(), av);

    let delete_item_input = DeleteItemInput {
        table_name: "session".to_string(),
        key: key,
        ..Default::default()
    };

    match dynamodb.delete_item(delete_item_input).sync() {
        Ok(_) => {
            debug!(applogger, "Deleted invalid session from ddb"; "sessid" => sessid);
        }
        Err(e) => {
            crit!(applogger, "Error in dynamodb"; "err" => e.to_string());
            panic!("Error in dynamodb: {}", e.to_string());
        }
    };
}

Lots, lots, lots more to do – but I’m having fun with this. It might go faster if I actually had a plan for what this web app actually does. I mean, I have an idea, but I haven’t mocked up a single page so I’m going slow and just enjoying hacking out Rust code as I go! Thanks for coming along with me!

Rust Functions, Modules, Packages, Crates, and You

Wooden pallets stacked one on top another
I know the code is in here… somewhere.

Come to find out, I’m learning Rust from old documentation. Both of the printed Rust books I have are for the pre-“2018 edition” and I think that’s contributing to some confusion I have about functions, modules, packages, and crates. A new version of the official book is coming out in the next month or so – I have a link to it through Amazon in the right sidebar. If you’ve been reading the online documentation, you’re ok – it is updated fir the “2018-edition”. I’ve looked at some of these parts of Rust before, but I recently found another new resource, the Edition Guide, which clears up some of my issues. Especially of interest here, is the section on Path Clarity which heavily influenced by RFC 2126 that improved this part of Rust.

I learned some of the history (and excitement) of RFC 2126 while listening to the Request for Explanation podcast, episode 10. Anyway, let’s go back to basics and have a look at Rust functions, modules, packages and crates as the language sits in mid-2019. I’ll present some examples from my web application we’ve been looking at. I’m going to cut out unnecessary bits to simplify things, so a “…” means there was more there in order for this to compile. You can always see whatever state it happens to be in, here.

Crates and Packages

A Rust crate (like Rocket or Diesel) is a binary or library of compiled code. A binary crate is runnable while a library crate is used for its functionality by being linked with another binary. A package (like my web app) ties together one or more crates with a single Cargo.toml file. The toml file configures the package‘s dependencies and some minimal information about compiling the source. A binary crate will have a src/main.rs with a main() function which directs how the binary runs. A library crate will have a src/lib.rs which is the top layer of the library. This top layer directs which pieces inside are available to users of the library.

Rust Functions

Functions are easy – subroutines in your source code. A function starts with fn, possibly receives some parameters and might return a value. Also, a function may be scoped as public or kept private. The main() function inside src/main.rs is a special function that runs when the binary is called from the command line. It dictates the start of your program and you take control from there. You may create other functions, just avoid reserved words (or use the r# prefix to indicate you mean YOUR function, not the reserved word, for instance r#expect if you want to name a function “expect”). Very similar to functions, are methods and traits, which we’ve looked at before.

<src/lib.rs>

...
use diesel::prelude::*;
...
pub fn setup_db() -> PgConnection {
    PgConnection::establish(&CONFIG.database_url)
        .expect(&format!("Error connecting to db"))
}

setup_db() is a fairly simple function – it accepts no incoming parameters and returns a database connection struct called PgConnection. It has pub before fn to indicate it is a “public” function. Without that, my web application bin/src/pps.rs could not call this function – it would not be in scope. Without pub, setup_db() would only be callable from within src/lib.rs. Since I am designing my application as a library crate, I choose to put setup_db() in the main src/lib.rs file. My binary that I will use to “run” my web application is in src/bin/pps.rs and contains a main() function.

Let’s look at the return type, PgConnection. This is a struct defined by the database ORM library crate, Diesel. The only way I could write a function that returns this particular type of struct is because I have use diesel::prelude::*; at the top (and it’s in the toml file as well). The Diesel library crate provides prelude as a simple way to bring in all Diesel has to offer my package. Diesel provides the PgConnection struct as public (or what good would the crate be), so I can now use that struct in my code. This also gives me the (method or trait, how can you tell?) establish(). Just like you’d call String::new() for a new string, I’m calling PgConnection::establish() for a new database connection and then returning it (see, no trailing ; on the line).




Rust Modules

Functions (and other things) can be grouped together into a Module. For instance, setup_logging() is also in src/lib.rs. However, I could have wrapped it inside a named module, like so:

<src/lib.rs>

...
pub mod setting_up {
    ...
    use logging::LOGGING;
    use settings::CONFIG;

    pub fn setup_logging() {
        let applogger = &LOGGING.logger;

        let run_level = &CONFIG.server.run_level;
        warn!(applogger, "Service starting"; "run_level" => run_level);
    }
}

Now it is part of my setting_up module. Here also, the module needs to be pub so that my application can use it and the public functions inside it. Now all of the enums and structs and functions inside the module setting_up are contained together. As long as they are public, I can still get to them in my application.

Notice I use logging::LOGGING; and use settings::CONFIG; These bring in those two structs so I can use the global statics that are built when then the application starts. I included pub mod logging; and pub mod settings; at the top level, in src/lib.rs, so they are available anyplace deeper in my app. I just need to use them since I reference them in this module’s code.

Splitting firewood with an axe

Split, for Clarity

On the other hand, instead of defining a module, or multiple modules, inside a single file like above, you can use a different file to signify a module. This helps split out and separate your code, making it easier to take in a bit at a time. I did that here, with logging.rs:

<src/logging.rs>

...
use slog::{FnValue, *};

pub struct Logging {
    pub logger: slog::Logger,
}

pub static LOGGING: Lazy<Logging> = Lazy::new(|| {
    let logconfig = &CONFIG.logconfig;

    let logfile = &logconfig.applog_path;
    let file = OpenOptions::new()
        .create(true)
        .write(true)
        .truncate(true)
        .open(logfile)
        .unwrap();

    let applogger = slog::Logger::root(
        Mutex::new(slog_bunyan::default(file)).fuse(),
        o!("location" => FnValue(move |info| {
        format!("{}:{} {}", info.file(), info.line(), info.module(), )
                })
        ),
    );

    Logging { logger: applogger }
});

I have a struct and a static instance of it, both of them public, defined in logging.rs. logging.rs becomes a module of my library crate when I specify it. At the top of src/lib.rs I have pub mod logging; which indicates my library crate uses that module file logging.rs and “exports” what it gets from that module as public (so my bin/src/pps.rs application can use what it provides).

In this case, you also see I use slog::{FnValue, *}}; which is like use slog::FnValue; (which I need for the FnValue struct) and use slog::*; which gives me the fuse struct and the o! macro. I was able to combine those into a single use statement to get just what I needed from that external crate.

The old books I have been referencing have you declaring the third-party crates you want to use in your application in your Cargo.toml file (which is still required), but also you’d have to bring each one in with an extern crate each_crate; at the top of main.rs or lib.rs. Thankfully, that’s no longer needed… 99% of the time. In fact, I had a long list of those myself – I am surprised cargo build didn’t warn me it was unneeded. Actually, I do have one crate I am using which still needs this “2015-edition” requirement: Diesel. Apparently, it is doing some fancy macro work and/or hasn’t been upgraded (yet?) for the “2018-edition” of Rust, so at the top of src/lib.rs, I have:

#[macro_use]
extern crate diesel;

A Few Standards and TOMLs

The Rust crate std is the standard library, and is included automatically. The primitive data types and a healthy list of macros and keywords are all included. But, if you need filesystem tools: use std::fs; and if you need a HashMap variable, you’ll need to use std::collections::HashMap; And yes, all external crates you depend on inside your source will need to be listed in Cargo.toml. This configuration helps you though – it updates crates automatically as minor versions become available, but does NOT update if a major version is released. You will need to do that manually, so you can test to see if the major release broke anything you depended on in your code. Here is a piece of my ever-growing Cargo.toml file for the web application so far:

...
[dependencies]
slog = "2.5.0"
slog-bunyan = "2.1.0"
base64 = "0.10.1"
rand = "0.7.0"
rand_core = "0.5.0"
rust-crypto = "0.2.36"
config = "0.9.3"
serde = "1.0.94"
serde_derive = "1.0.94"
serde_json = "1.0.40"
once_cell = "0.2.2"
dotenv = "0.14.1"
chrono = "0.4.7"
rocket = "0.4.2"
rocket-slog = "0.4.0"

[dependencies.diesel]
version = "1.4.2"
features = ["postgres","chrono"]

[dependencies.rocket_contrib]
version = "0.4.2"
default-features = false
features = ["serve","handlebars_templates","helmet","json"]

Better Logging for the Web Application

I replace log and simple_logger with slog and sloggers, and then change again

Part of a Series: Designing a Full-Featured WebApp with Rust
Part 1: Piecing Together a Rust Web Application
Part 2: My Next Step in Rust Web Application Dev
Part 3: It’s Not a Web Application Without a Database
Part 4: Better Logging for the Web Application
Part 5: Rust Web App Session Management with AWS
Part 6: OAuth Requests, APIs, Diesel, and Sessions
Part 7: Scraping off the Dust: Redeploy of my Rust web app
Part 8: Giving My App Secrets to the AWS SecretManager

A journal ledger of accounts... like logging but only uses numbers
See… structured logging…

When I last left my sample web application, I was doing simple logging to the terminal which is not useful for very long. I want structured logging and I want the app and Rocket to write to (separate) files. So, let’s switch out the crates log and simple_log for slog and sloggers to get better logging for my web application.

Application Logging

Slog‘s “ambition is to be The Logging Library for Rust” and it sure seems adaptable. There are several “helper” crates to log to the terminal or syslog, log as json, etc. However, slog is complex! Which seems to be the main reason sloggers came about. Sloggers brings you the most useful features of slog without the complex setup. It happens to include an example of configuring via an inline TOML string. I’ve already got my CONFIG global pulling from a TOML file, so I add this to conf/development.toml:

<conf/development.toml>

...
[logconfig]
type = "file"
format = "compact"
source_location = "module_and_line"
timezone = "local"
level = "debug"
path = "log/error.log"
rotate_size = 1073741824
rotate_keep = 2

[webservice]
weblog_path = "log/access.log"
...

Plus, I add logging.rs to set up the global LOGGING instance so everyone can log; just like everyone can pull from the global CONFIG instance:

<logging.rs>

use crate::settings::CONFIG;
use crate::sloggers::{Build, Config};
use once_cell::sync::Lazy;

#[derive(Debug)]
pub struct Logging {
    pub logger: slog::Logger,
}

pub static LOGGING: Lazy<Logging> = Lazy::new(|| {
    let logconfig = &CONFIG.logconfig;

    let builder = logconfig.try_to_builder().unwrap();
    let logger = builder.build().unwrap();

    Logging { logger: logger }
});

Sloggers is really easy to use to set up a slog instance. Here, I simply pull the logconfig from my global CONFIG, build the logger, and store it in my new OnceCell Lazy LOGGING global.




And then, Web Access Logging

A spider web, wet from rain...

At this point, application logging is going to my new log/error.log file, but Rocket logging is still coming to the screen. This is actually good – as I mentioned, I want to have them going to separate files anyway. So now with some slogging experience, it’s a simple matter to set up a second file, specifically for Rocket. I do need to add yet another crate, this time rocket_slog – so that Rocket internals can glom on to the slog instance that I make. Here is the new start_webservice function:

<src/lib.src>

...
pub fn start_webservice() {
    let logger = &LOGGING.logger;

    let weblog_path = &CONFIG.webservice.weblog_path;
    let bind_address = &CONFIG.webservice.bind_address;
    let bind_port = &CONFIG.webservice.bind_port;

    // start rocket webservice
    let version = include_str!("version.txt");

    let mut builder = FileLoggerBuilder::new(weblog_path);
    builder.level(Severity::Debug);
    let weblogger = builder.build().unwrap();
    let fairing = SlogFairing::new(weblogger);

    warn!(
        logger,
        "Waiting for connections..."; "address" => bind_address, "port" => bind_port, "version" => version);

    rocket::ignite()
        .attach(fairing)
        .attach(Template::fairing())
        .attach(SpaceHelmet::default())
        .mount("/", routes![routes::index, routes::favicon])
        .mount("/img", StaticFiles::from("src/view/static/img"))
        .mount("/css", StaticFiles::from("src/view/static/css"))
        .mount("/js", StaticFiles::from("src/view/static/js"))
        .launch();
}

Now I have log/error.log for application logging (I’ll rename that later to log/app.log) and log/access.log for Rocket logging. But I still don’t have structured logs!

Finally, Structured Logs

I try several ways (with my limited Rust knowledge and experience) but it appears that with sloggers ease of use, you trade away options if you want to get fancy. At least, I couldn’t figure out how to get slog_json or something called slog_bunyan to work with sloggers. It looks like I have to deal with slog directly.

Later, on another night, I tackle these changes. I dump sloggers and, though much searching around, I end up with a merge of two examples I find (from slog and slog-bunyan). Thankfully, I am getting a little better at understanding the API docs that are posted for crates! Anyway, here is how the app logging setup looks now:

<src/logging.rs>

...
pub static LOGGING: Lazy<Logging> = Lazy::new(|| {
    let logconfig = &CONFIG.logconfig;

    let logfile = &logconfig.applog_path;
    let file = OpenOptions::new()
        .create(true)
        .write(true)
        .truncate(true)
        .open(logfile)
        .unwrap();

    let applogger = slog::Logger::root(
        Mutex::new(slog_bunyan::default(file)).fuse(),
        o!("location" => FnValue(move |info| {
        format!("{}:{} {}", info.file(), info.line(), info.module(), )
                })
        ),
    );

    Logging { logger: applogger }
});

Notice I still pull the applog_path from the CONFIG and then create that filehandle. Next, a single but complicated call into slog returns a Logger instance, which I store into my global. The o! macro, provided by the slog crate, lets me add a JSON field to every log record that gets written. I copy from an example and add the file, line number, and module name where the log was generated. I’ll probably come back to this later and add more data. Also, notice I am using slog_bunyan for the JSON formatting. slog-json actually recommends using slog_bunyan for a “more complete output format.” Bunyan logging seems to originate as a node.js module. Lastly, I also change the weblogger in the same way – now both are JSON logs and still log to separate files.

This actually wasn’t as complicated as I feared. I’ve lost some of the easily configurable sloggers features like auto file rotation, but I’ll figure that out as well – it’s probably actually just a slog feature that I need to enable.

Results, JSON Style

Here is how both look now – I think next post I’ll work on a JSON log viewer so we can watch these easier, as we develop.

<log/app.log>

{"msg":"Service starting","v":0,"name":"slog-rs","level":40,"time":"2019-07-20T03:40:13.781382592+00:00","hostname":"ip-169-254-169-254","pid":6368,"location":"src/lib.rs:48 ppslib","run_level":"development"}
{"msg":"Waiting for connections...","v":0,"name":"slog-rs","level":40,"time":"2019-07-20T03:40:13.784899718+00:00","hostname":"ip-169-254-169-254","pid":6368,"location":"src/lib.rs:76 ppslib","version":"0.1.0","port":3000,"address":"0.0.0.0"}

<log/access.log>
{"msg":"listening","v":0,"name":"slog-rs","level":30,"time":"2019-07-20T05:31:21.068959173+00:00","hostname":"ip-169-254-169-254","pid":7700,"address":"http://0.0.0.0:3000"}
{"msg":"request","v":0,"name":"slog-rs","level":30,"time":"2019-07-20T05:31:22.402815352+00:00","hostname":"ip-169-254-169-254","pid":7700,"uri":"\"/\"","method":"Get"}
{"msg":"response","v":0,"name":"slog-rs","level":30,"time":"2019-07-20T05:31:22.404425952+00:00","hostname":"ip-169-254-169-254","pid":7700,"status":"200 OK","route":"\u001b[32mGET\u001b[0m \u001b[34m/\u001b[0m \u001b[36m(\u001b[0m\u001b[35mindex\u001b[0m\u001b[36m)\u001b[0m"}

Again, if all that was complicated to follow and you prefer to look at the git commits as I went through this ordeal, check out the repository.