I started the I2Cdevlib project in 2010 as a bit of yak-shaving intended to help with my Keyglove project, which at the time required access to nearly half a dozen I2C peripherals. It was a new protocol to me, but I saw value in creating an extensible device library rather than the independent and somewhat messy per-device implementations I’d seen.
My happy little repository has grown to a reasonably well-known project with 4,868 forks as of the time of this post, thanks to the continuing interest of 58 contributors. I have the MPU-6050 IMU from InvenSense (now TDK) to credit for much of this interest, but others had an impact as well.
However, I didn’t think ahead enough to set the project on a path that would cope with the level of interest that I2Cdevlib has since seen. I built the low-level interface code specifically on top of the Arduino Wire library, and specifically for register-based sensor devices. This has turned out to be severely limiting in certain cases, and to cause general headaches in many others.
Shortcomings Laid Bare
No doubt many developers who have used the I2Cdevlib code could point out numerous flaws in the project design. I’m here to concur wholeheartedly. Don’t get me wrong–there’s still a lot about the project that I like. But it would be a sad commentary on my evolving skillset if I thought everything I wrote eight years ago was still A-OK. Plus, there have been great improvements to GitHub during that time, and some fantastic new tools on the scene like Travis CI, which wasn’t around at all when I started.
Here’s what I want to do better with this time.
Fix #1: Repository Structure
This is a big one. Although the Arduino IDE’s library manager wasn’t a thing in 2010, my choice to put everything into one repo has made it impossible to take advantage of it today. Actually, the original structure had only Arduino-specific code in it, without top-level folders for specific platforms. I decided to change this early in the project’s life, but instead of creating new repositories as I should have, I just moved stuff around and made the original repo more complicated. Oops.
In addition to preventing integration with almost any popular GitHub-aware IDE library manager available today, this also means every reported issue is in one giant bucket. There’s no logical separation for problems with the Arduino implementation of the MPU-6050 device code and the ESP32 implementation of the core I2Cdev hardware layer code. It’s a mess.
This also means that anyone using the repo will necessarily have to pull down a huge amount of code that they will never use. Most people want the core hardware layer code and one or maybe two device libraries, only for the platform they’re using. Too bad! You have to download the entire fileset and move or copy only the pieces you need into your project or library folder.
A monolithic structure also means there is no meaningful way to do commit-tagged versioning for official releases. The various sub-projects inside the repo are independent of each other, and it doesn’t make sense to call the entire thing “1.0.5” or whatever.
The solution is to use a more flexible structure with a bunch of individual repositories for each logically separate component. For example:
- Arduino core hardware access layer
- Arduino MPU-6050 library
- Arduino ADXL345 library
- RasPi core hardware access layer
- RasPi MPU-6050 library
- RasPi ADXL345 library
- nRF52 core hardware access layer
…and so on.
These will also be under a GitHub organization account dedicated to this project, rather than my own personal account. It just makes more sense. Yes, we’ll end up with many more individual repos, but that’s not a problem. In any case, it’s a tiny price to pay for the benefits such a structure allows, including fixes for all of the problems mentioned above:
- Easy integration into library managers like the Arduino IDE and PlatformIO
- Per-library automated testing with Travis CI
- Per-library versioning and release tracking
- Per-library issue reporting
- Non-wasteful installation, no more cloning a majority of personally useless stuff
- Easy future expansion to new platforms simply by adding new repos
It will also make it easier for specific contributors to “own” libraries that they commit or happen to be good at and interested in supporting. Trust me when I say nobody is going to fully grasp every aspect of every subproject on every platform in this kind of a repo, unless that’s their full-time job. But someone might be happy to assume responsibility of a library or two that they created and/or use regularly.
Fix #2: Hardware Access Layer Code
This is another big one. Using the Arduino Wire library (of 2010) as the foundation for everything else has made it practically impossible to add support for multiple I2C peripheral objects, timeouts, and some lower-level I2C functions. It also precludes the use of any other common peripheral protocols like SPI, UART, and 1-Wire, which is something I’ve wanted to do in some fashion for a very long time. For example, the SPI-supporting MPU-6000 sensor allows much faster data speeds than the I2C-only MPU-6050, while using virtually the same protocol. There’s no good reason the same library collection can’t support both.
Further, the current architecture provides no easy way to independently choose which I2C implementation (Wire, NBWire, Fastwire, etc.) you want to use. You can switch by commenting or uncommenting certain #define statements in the I2Cdev core header, but this is unwieldy and non-obvious. All of the different implementations are bundled together in the same source files, which only gets worse as new implementations are added.
The solution to both of these problems is to build reusable interface objects that are assigned to core and device instances when they are created. For example, on the Arduino Due platform, you could theoretically drive four MPU-6050 devices using the “Wire” and “Wire1” objects and both slave addresses on each set of two IMUs. You need only to be able to pass in the object you want to use.
For example, here’s some pseudocode for it how might work for two Wire-based HAL instances powering four MPU-6050 IMUs:
HalCore i2c0(&Wire);
HalCore i2c1(&Wire1);
MPU6050 imu0(&i2c0, 0x68);
MPU6050 imu1(&i2c0, 0x69);
MPU6050 imu2(&i2c1, 0x68);
MPU6050 imu3(&i2c1, 0x69);
With the correct class inheritance and abstraction, the top-layer “MPU6050” objects won’t either know or care what the underlying communication layer is. You could swap in NBWire instances for the “HalCore” objects just as easily.
This implementation is not really more complicated than the original approach, but it has far more flexibility. Admittedly, supporting this on C-only platforms is not as straightforward due to the lack of classes and inheritance, but it isn’t impossible.
Fix #3: Device Definitions
The web-based database-driven register map and transaction analyzer are my favorite things about the original project, and they’re still online. The concept is great, but the implementation has turned out to be too inaccessible. Because of this, what could have been the most valuable aspect of this whole project has languished because I never quite managed to expose the creation/modification interface to the people most likely to do great things with it.
To solve this problem and some related problems with version control of device definitions, I’ve decided to simply move everything into JSON files inside their own separate repo. Everything that was stored in the database can just as easily exist in this format, and this allows much greater transparency and the opportunity for easy new device contributions. It also provides a simple way for external projects to directly access device data for other purposes, and it makes it possible for Travis CI to automatically build code and documentation whenever a new device is created or an existing one is modified.
The device definitions will be the heart of the whole project, responsible for a significant portion of code generation and required before any supporting repos, libraries, or examples can be submitted.
As of the time of this post, I’m in the process of creating JSON schema definitions which will be used both for validation and for automatic form generation using the react-jsonschema-form plugin. This mechanism is clean, functional, and as easy as it possibly could be given the requirements. It also remains flexible if the schema gets new elements in the future, which will undoubtedly happen.
The schema will be a work-in-progress for a while, but here are the known requirements so far:
- Unique device identification slugs
- Unique manufacturer identification slugs
- IC/module package and pin definitions for reference
- I2C address details (for I2C devices)
- Multiple protocol support (I2C, SPI, UART, etc.)
- Multiple simultaneous protocol variants (e.g. BNO080 IMU supports two different types of UART communication)
- Byte ordering definition at device level and register level
- Register-based protocol definitions (e.g. MPU-6050 IMU register map, structure, and multi-byte R/W behavior)
- Stream-based rigid binary protocol definitions (e.g. BLE112 module BGAPI protocol)
- Stream-based flexible text protocol definitions (e.g. WT12 module iWRAP protocol)
- Information about whether external pin signals are supported or required (e.g. interrupt, wake)
My goal is to create the schema such that it can be extended to support new features when needed without breaking backwards compatibility. I know from experience that this is hard to do perfectly, but at least we’ve got nearly eight years of “wouldn’t-it-be-nice-if-we-could-only” experience.
Fix #4: Hardware Analyzer Integration
Here’s an area where a properly built peripheral library can shine, and where I always hoped to take I2Cdevlib. With an open and accessible collection of device and structure/protocol definitions, and a logic analyzer or even a passive sniffer like the Bus Pirate, it should be completely possible to have a simple Python GUI that spits out live, annotated analysis of peripheral transactions. Think Wireshark but for I2C/SPI/UART peripherals. This would not only be phenomenal for library development, but also great for troubleshooting.
I’ll one-up myself here: with a cheap ATSAMD21-based MCU (not even Bus Pirate levels of financial expenditure) and a USB CDC interface, and a Chrome browser that supports the WebUSB specification, you could even run such a tool right in the browser. Note that WebUSB is supported in the Android version of Chrome. This means that you could have a powerful and portable protocol sniffer/analyzer with a cheap smartphone and $10 of external hardware running via USB OTG.
(Yes, it might be easier to do the legwork with a real mobile app than trying to leverage Chrome, and I’m definitely open to that. There are some nice conveniences with a browser-only implementation though.)
Think about this. With a bit of back-end data capture syncing, you could walk your smartphone and OTG analyzer over to a running device, grip RX, TX, and GND pins with spring clip probes, and just leave it there while you walk back to your desk and watch the data come in on the easier-to-work-with desktop version of the capture/analyzer website. I’m already salivating.
But wait, there’s more! Reverse-engineering protocols is tedious. Even just translating a published specification from an API reference manual into source code (or JSON data, as the case may be) is mind-numbing work. The good news is that many protocols follow similar structural patterns, because there’s no reason to reinvent the wheel. With some intelligent guesswork, an analyzer tool could present some possible structural foundations to start from, and overlay them on top of a data capture for quick visual review. Start-of-frame bytes? Checksums? Type, length, or sequence bytes in a header? Pick the combination that seems to fit, then tweak the parameters until there are no misalignments.
The front end of this would require some fancy programming in React (or similar), which admittedly I’m a bit rusty on, but the concept is not hard to grasp. I can see all of the moving pieces of this project working in my head. I know it’s technically possible. But it is a big project.
Who’s with me?