(find . -name '*.rs' -not -path '*target*' -not -path '*proto*' | xargs cat) \ | wc -l 3104 (find . -name '*.js' -not -path '*target*' -not -path '*node_modules*' | xargs cat) \ | wc -l 563
My overall impression of Rust is that it is pleasant to work with once you grok the memory model. Despite the language being low level, I only had two panics during development. I felt comfortable in the same way I do when developing Scala (and that I don't in Ruby or Golang): if the code compiles, it is likely to be correct.
The borrow checker was a new concept for me. My previous implementation of this game was an evented ruby implementation which was (ahem) cavalier in how it shared object references. I initially fought with the compiler a lot, but some practice helped me get into my head when to use references, when to
clone, and when to
move. I tried to read some tutorials on borrowing and none of it stuck. I had to get comfortable by doing.
The most practical consequence of Rust's memory safety is that there is no
null. Having this baked into the language (and the standard library!) is liberating. Whole classes of bugs vanish because using
Result is not only the default, but required.
While not part of rustc proper, Clippy is a rustup component that acts as a linter. I found Clippy to be incredibly useful while learning the language. Its large library of lints helped me to internalize how to write idiomatic Rust. Early on, I added
#![warn(clippy::all, clippy::pedantic)] to my crate.
Clippy helped me write more performant code by suggesting the correct combinators to use, e.g.
map_or instead of
map(...).unwrap_or_else(...), and avoiding unnecessary clones by using references when functions did not consume arguments.
Clippy required nightly to analyze all of the crates that Punchtop depended on, which was initially off-putting. Requiring nightly also interacted poorly with the
protobuf-codegen-pure build dependency: the codegen only output
#![allow(clippy)] in the generated build files, which was incompatible with nightly Clippy. I wrote a codemod in
build.rs to work around this (I wish I could have found a library that did this for me).
Clippy offers the ability to selectively disable lints by attaching an
#[allow(clippy::...)] to any code unit. I had to do this in a few places: generated protobufs, some serde enums, and rocket route functions. Selectively disabling lints was not hard, but it was non-obvious.
The most frustrating part of using Clippy is that it did not force a reanalysis of all packages in my cargo workspace when invoked multiple times. A (maybe bad?) habit I learned from rubocop is to incrementally fix lint errors and rerun the linter to see what I have left. Clippy does not force a re-compile of source packages in the current workspace requiring contortions like
touch src/main.rs to get the updated lint errors.
Overall, I found Clippy to be quite excellent and it made my Rust code much better.
My initial implementation of Punchtop used rodio as the audio output device. Once I added the Chromecast sink, the shared API for these two modules evolved rapidly (especially once the API started to get infected by async). I found that the rodio implementation was slowing me down too much, so I deleted it. Looking back, a better solution would have been to add a
#[cfg(...)] incantation to disable compilation for the rodio-based module.
I have some tests! which is more than I can normally say about a side project. I found it much easier to write tests for a few reasons:
#[test]is native to rustc, so I did not have to mess with choosing a unit test framework and getting it integrated with my project.
The only non-obvious part of writing tests was wrapping test functions in a
mod test that is conditionally compiled during test builds with
#[cfg(test)]. The Rust book section on testing has examples that show to do this, but never specifies why to do it. The answer is decreased compilation time in non-test builds.
cargo doc is excellent. I ran this on an airplane and was able to develop with no Internet access.
The stream-util module I wrote to gracefully drain a futures mpsc channel is the most well-documented module I have ever written. I added a directive to fail builds on missing docs. I modeled the docs on those in BurntSushi/walkdir and also ported them over to the README for the crate. Like I said above, doc tests are amazing.
Linking to code units in rustdoc was hard to do correctly since there were so many ways to do it. It took me more than a few attempts to get the resolution rules internalized.
When porting the rustdoc to the README, I had to manually insert links that resolved the modules in external crates (mostly tokio) that were pertinent to using the library.
My biggest frustration with reading documentation was using the Rust book(s) from Google. The books themselves are excellent, but the SEO is terrible. There are multiple versions linked in Google. The books know that I want the most recent version because they all link to it, but I'd prefer this be handled via a redirect so Google knows what documentation is current.
Cargo is the Yarn equivalent for Rust. It handles managing dependencies, building your code, and running your code. Cargo is very pleasant to work with. Pulling in libraries was effortless (even git dependencies), as was running code in either debug or release configurations.
Rust 2018 Edition came out early on when developing Punchtop and Cargo made it easy to upgrade to the latest supported idioms.
One thing I missed from Yarn was an
outdated command that enumerates libraries that have available updates. I supported this part of my workflow with the [cargo-outdated] crate which does the job admirably.
As Punchtop grew more complex, I had several trees within
src that were shaped like independent libraries. I split my monolithic
punchtop crate into multiple crates within a shared cargo workspace.
I also found that splitting libraries out into their own crates made it easier to reason about the visibility of structs and functions without having to mess with more exotic visibility modifiers.
When I split Punchtop into multiple crates, I missed that I needed to add my Clippy
!#[warn(...)] directive to the
lib.rs of the new crates.
Rust has many good libraries. I was able to offload lots of tricky logic to crates. For example, I do not want to be in the business of parsing an mp4 to find out if it is at least 60 seconds long.
Tokio is a suite of libraries for writing async code. It is big, but was mostly easy to use. These are the highlights.
tokio crate is composed of many sub crates, e.g.
tokio-codec. I initially had a mix of these included in Punchtop and was confused by the differing version numbers. Ultimately I learned that tokio recommends depending on
tokio in crates with a
main.rs and the sub crates in crates with a
lib.rs. I used this newfound knowledge to trim 48 dependencies from the
tokio-codec is my favorite part of Tokio.
tokio-codec enables reading and writing frames to a socket. The cast protocol consists of
u32-length prefixed protobuf frames so this was a perfect fit. It was very easy to implement a stateful decoder. My encoder consumed a high-level playback command enum and the decoder produced decoded protobufs which were handled by the channel multiplexer. The separation of concerns made it clear where to check protocol invariants such as maximum payload size.
The one gotcha of
tokio-codec was the
bytes crate. I was missing a call to
buf.reserve(len), which was the source of one of my panics.
Much has been written about the
futures crate. The two things I found helped me the most when using it are
impl trait and
IntoFuture combined with generics. I would like to invest more in using named
Futures to clean up the
cast-client public API.
web_view provides high-level Rust bindings for the cross-platform webview single-header C library. The library allows two-way communication to JS running in the webview using stringified JSON. I built a React and Redux app and a minimal amount of MVC glue to track application state in Rust. The result was an easy to extend UI. I even integrated the webpack build into the build script for
My one complaint is that
RUST_LOG=debug cargo run dumps all of the verbose debug logs from cargo while your program is being built.
Rocket is a hyper-based HTTP framework. Rocket requires nightly, but is very ergonomic to use. It requires less ceremony than even a Sinatra server. The statically-typed route functions were pleasant to work with.
I did have to jump through some hoops to use Rocket as an embedded (vs. application) server, including launching rocket in a thread instead of scheduling it on my tokio reactor, discovering the bind interface, and configuring an unused secret key.
mp3_duration is a single-purpose crate did exactly what I needed it to do until it panicked on some of my MP3s. Since I was using this crate to filter items out of a playlist, I isolated it using
panic::catch_unwind and ignored files it could not parse. I should probably find a panicking sample MP3 and file a GitHub issue.
Serde is my favorite of all the crates I used in Punchtop. I have previous experience with statically-typed JSON libraries in Jackson and json4s. Serde is by far the easiest JSON library to work with.
Serde uses Rust macro magic to derive encoders and decoders directly from structs. The encoder and decoder are configurable with a few other macros. My two favorites are remapping rustfmt-compliant struct fields to the native JSON camel case style and field-tagged enums AKA switching deserialization based on a type tag in the JSON. Field-tagged enums gives you all the benefits of Rust enums when dealing with JSON, most notably ensuring match arms are exhaustive.
At this point in the project, I feel reasonably comfortable working with Rust, but there are still lots of things left to learn. My next goal is to write a native Cocoa frontend for Punchtop which means either using the
cocoa crates or embedding
cast-client into a Swift app via FFI. Either way, I am excited to make more progress.