diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/.nvmrc | 1 | ||||
| -rw-r--r-- | docs/blog/2020-10-03-bootloader-fix.md | 195 | ||||
| -rw-r--r-- | docs/docs/behavior/outputs.md | 59 | ||||
| -rw-r--r-- | docs/docs/dev-guide-new-shield.md | 15 | ||||
| -rw-r--r-- | docs/docs/dev-guide-usb-logging.md | 26 | ||||
| -rw-r--r-- | docs/docs/feature/keymaps.md | 4 | ||||
| -rw-r--r-- | docs/docs/intro.md | 2 | ||||
| -rw-r--r-- | docs/docs/user-setup.md | 7 | ||||
| -rw-r--r-- | docs/package.json | 5 | ||||
| -rw-r--r-- | docs/sidebars.js | 1 |
10 files changed, 310 insertions, 5 deletions
diff --git a/docs/.nvmrc b/docs/.nvmrc new file mode 100644 index 0000000..1a2f5bd --- /dev/null +++ b/docs/.nvmrc @@ -0,0 +1 @@ +lts/*
\ No newline at end of file diff --git a/docs/blog/2020-10-03-bootloader-fix.md b/docs/blog/2020-10-03-bootloader-fix.md new file mode 100644 index 0000000..8a9fd7f --- /dev/null +++ b/docs/blog/2020-10-03-bootloader-fix.md @@ -0,0 +1,195 @@ +--- +title: Fixing the Mysterious Broken Bootloader +author: Nick Winans +author_title: Contributor +author_url: https://github.com/Nicell +author_image_url: https://avatars1.githubusercontent.com/u/9439650 +tags: [bootloader, keyboards, firmware, oss, ble] +--- + +Recently I was able to fix the "stuck in the bootloader" issue in +[#322](https://github.com/zmkfirmware/zmk/pull/322) that had been plaguing us +for quite some time. I want to go over what the issue was, how the issue was +diagnosed, and how it was fixed. + +## Background + +What exactly is the "stuck in the bootloader" issue? Seemingly randomly, users' +keyboards would suddenly stop working and when they would reset their keyboard +they would get put into the bootloader instead of back into the firmware. This +would require the user to re-flash the firmware again to get into the firmware. +That wouldn't be so bad except for the fact that once this occurs, every reset +would require the user to re-flash the firmware again. The only way to really +fix this issue was to re-flash the bootloader itself, which is a huge pain. + +Going into this, all we knew was that this issue was most likely introduced +somewhere in the [#133](https://github.com/zmkfirmware/zmk/pull/133), which +added Bluetooth profile management. We've had quite a few attempts at trying to +recreate the issue, but we never were able to get it to happen consistently. + +## Diagnosing the issue + +This issue had been happening sporadically for the past month, and I finally +decided to dig in to see what was going on. We started in the Discord and +discussed what was common between all of the people who have experienced this +issue. Everyone who had this issue reported that they did quite a bit of profile +switching. This lined up with the possible connection to the Bluetooth profile +management pull request. + +### Pinpointing the cause + +I had a hunch that this was related to the settings system. The settings system +is used by profile Bluetooth switching, and the settings system works directly +with the system flash. Based on this hunch, I tried spamming the RGB underglow +cycle behavior on my main keyboard. Sure enough after a couple minutes, I got +stuck in the bootloader. I was even able to reproduce it again. + +This was an important discovery for two reasons. First, I was able to recreate +the issue consistently, which meant I could set up logging and more closely +monitor what the board was doing. Second, this more or less proved that it was +specifically the settings system at fault. Both Bluetooth profile switching and +RGB underglow cycling trigger it, and the one common piece is they save their +state to settings. + +### Settings system overview + +To understand what's going wrong, we first need to understand how the settings +system works. Here's a diagram to explain the flash space that the settings +system holds for our nRF52840 based boards (nice!nano, nRFMicro, BlueMicro). + + + +The settings flash space lives at the end of the flash of the chip. In this case +it starts at `0xF8000` and is `0x8000` bytes long, which is 32KB in more +comprehensible units. Then due to the chip's architecture, this flash space is +broken into pages, which are `0x1000` bytes in size (4KB). + +The backend that carries out the settings save and read operation in ZMK is +called NVS. NVS calls these pages sectors. Due to how flash works, you can't +write to the same bytes multiple times without erasing them first, and to erase +bytes, you need to erase the entire sector of flash. This means when NVS writes +to the settings flash if there's no erased space available for the new value, it +will need to erase a sector. + +### Logging discoveries + +So first I enabled logging of the NVS module by adding +`CONFIG_NVS_LOG_LEVEL_DBG=y` to my `.conf` file. I repeated the same test of +spamming RGB underglow effect cycle and the resulting logs I got were this: + +``` +[00:00:00.000,671] <inf> fs_nvs: 8 Sectors of 4096 bytes +[00:00:00.000,671] <inf> fs_nvs: alloc wra: 3, f70 +[00:00:00.000,671] <inf> fs_nvs: data wra: 3, f40 +// A bunch of effect cycle spam +[00:02:34.781,188] <dbg> fs_nvs: Erasing flash at fd000, len 4096 +// A bunch more effect cycle spam +[00:06:42.219,970] <dbg> fs_nvs: Erasing flash at ff000, len 4096 +// A bunch more effect cycle spam +// KABOOM - bootloader issue +``` + +So at start up, we can see that the 8 sectors of 4KB are found by NVS properly, +however, I wasn't sure what the second and third lines meant, but we'll get back +to that. Nonetheless the next two logs from NVS showed erasing the sector at +`0xFD000` and then erasing the `0xFF000` sector. + + + +It's really odd that the third to last sector and the last sector are erased, +and then shortly after the bootloader issue is hit. I really had no explanation +for this behavior. + +### Reaching out to Zephyr + +At this point, I nor anyone else working on the ZMK project knew enough about +NVS to explain what was going on here. [Pete +Johanson](https://github.com/petejohanson), project founder, reached out on the +Zephyr Project's Slack (ZMK is built on top of Zephyr if you weren't aware). +Justin B and Laczen assisted by first explaining that those `alloc wra` and +`data wra` logs from earlier are showing what data NVS found at startup. + +More specifically, `data wra` should be `0` when it first starts up on a clean +flash. As we can see from my earlier logging on a clean flash I was instead +getting `f40`. NVS is finding data in our settings sectors when they should be +blank! We were then given the advice to double check our bootloader. + +### The Adafruit nRF52 Bootloader + +Most of the boards the contributors of ZMK use have the [Adafruit nRF52 +Bootloader](https://github.com/adafruit/Adafruit_nRF52_Bootloader), which allows +for extremely easy flashing by dragging and dropping `.uf2` files onto the board +as a USB drive. Every bootloader takes up a portion of the flash, and in the +README explains that the first `0x26000` is reserved for the bootloader with the +nRF52840, and we've properly allocated that. + +However, there isn't a full explanation of the flash allocation of the +bootloader in the README. There's a possibility that the bootloader is using +part of the same flash area we're using. I reached out on the Adafruit Discord, +and [Dan Halbert](https://github.com/dhalbert) pointed me towards the [linker +map](https://github.com/adafruit/Adafruit_nRF52_Bootloader/blob/master/linker/nrf52840.ld) +of the nRF52840. Let's take a look. + +``` +FLASH (rx) : ORIGIN = 0xF4000, LENGTH = 0xFE000-0xF4000-2048 /* 38 KB */ + +BOOTLOADER_CONFIG (r): ORIGIN = 0xFE000 - 2048, LENGTH = 2048 + +/** Location of mbr params page in flash. */ +MBR_PARAMS_PAGE (rw) : ORIGIN = 0xFE000, LENGTH = 0x1000 + +/** Location of bootloader setting in flash. */ +BOOTLOADER_SETTINGS (rw) : ORIGIN = 0xFF000, LENGTH = 0x1000 +``` + +Here's a diagram to show this a bit better. + + + +We've found the issue! As you can see from the red bar (representing our +settings flash area), we've put the settings flash area _right on top_ of the +Adafruit bootloader's flash space. Oops! + +This also shines some light on why NVS erased `0xFD000` and `0xFF000` sectors. +It's possible there was no flash written to `0xFD000` because the bootloader +didn't use up all of that space it has, and then there possibly weren't any +bootloader settings set yet, so `0xFF000` could be used and erased by NVS too. + +After erasing `0xFF000`, NVS probably next erased a rather important part of the +bootloader that resulted in this issue at hand. In my opinion, we're pretty +lucky that it didn't delete an even more vital part of the bootloader. At least +we could still get to it, so that we could re-flash the bootloader easily! + +## The solution + +Now that we've found the issue, we can pretty easily fix this. We'll need to +move the settings flash area back so that it doesn't overlap with the +bootloader. First we calculate the size of the of flash area the bootloader is using. + +``` +0x100000 (end of flash) - 0x0F4000 (start of bootloader) = 0xC000 (48KB) +``` + +So the bootloader is using the last 48KB of the flash, this means all we need to +do is shift back the settings area and code space `0xC000` bytes. We'll apply +this to all of the `.dts` files for the boards that were affected by this issue. + +```diff + code_partition: partition@26000 { + label = "code_partition"; +- reg = <0x00026000 0x000d2000>; ++ reg = <0x00026000 0x000c6000>; + }; + + +- storage_partition: partition@f8000 { ++ storage_partition: partition@ec000 { + label = "storage"; +- reg = <0x000f8000 0x00008000>; ++ reg = <0x000ec000 0x00008000>; + }; +``` + +And with those changes, we should no longer run into this issue! In the process +of these changes, we lost 48KB of space for application code, but we're only +using around 20% of it anyways. 🎉 diff --git a/docs/docs/behavior/outputs.md b/docs/docs/behavior/outputs.md new file mode 100644 index 0000000..ae81249 --- /dev/null +++ b/docs/docs/behavior/outputs.md @@ -0,0 +1,59 @@ +--- +title: Output Selection Behavior +sidebar_label: Output Selection +--- + +## Summary + +The output behavior allows selecting whether keyboard output is sent to the +USB or bluetooth connection when both are connected. This allows connecting a +keyboard to USB for power but outputting to a different device over bluetooth. + +By default, output is sent to USB when both USB and BLE are connected. +Once you select a different output, it will be remembered until you change it again. + +## Output Command Defines + +Output command defines are provided through the [`dt-bindings/zmk/outputs.h`](https://github.com/zmkfirmware/zmk/blob/main/app/include/dt-bindings/zmk/outputs.h) +header, which is added at the top of the keymap file: + +``` +#include <dt-bindings/zmk/outputs.h> +``` + +This allows you to reference the actions defined in this header: + +| Define | Action | +| --------- | ----------------------------------------------- | +| `OUT_USB` | Prefer sending to USB | +| `OUT_BLE` | Prefer sending to the current bluetooth profile | +| `OUT_TOG` | Toggle between USB and BLE | + +## Output Selection Behavior + +The output selection behavior changes the preferred output on press. + +### Behavior Binding + +- Reference: `&out` +- Parameter #1: Command, e.g. `OUT_BLE` + +### Examples + +1. Behavior binding to prefer sending keyboard output to USB + + ``` + &out OUT_USB + ``` + +1. Behavior binding to prefer sending keyboard output to the current bluetooth profile + + ``` + &out OUT_BLE + ``` + +1. Behavior binding to toggle between preferring USB and BLE + + ``` + &out OUT_TOG + ``` diff --git a/docs/docs/dev-guide-new-shield.md b/docs/docs/dev-guide-new-shield.md index 94f7d1d..e1d6f25 100644 --- a/docs/docs/dev-guide-new-shield.md +++ b/docs/docs/dev-guide-new-shield.md @@ -27,6 +27,15 @@ ZMK support for split keyboards requires a few more files than single boards to ## New Shield Directory +:::note +This guide describes how to add shield to the ZMK main repository. If you are building firmware for your +own prototype or handwired keyboard, it is recommended to use your own user config repository. Follow the +[user setup guide](./user-setup.md) to create your user config repository first. When following the rest +of this guide, replace the `app/` directory in the ZMK main repository with the `config/` directory in your +user config repository. For example, `app/boards/shields/<keyboard_name>` should now be +`config/boards/shields/<keyboard_name>`. +::: + Shields for Zephyr applications go into the `boards/shields/` directory; since ZMK's Zephyr application lives in the `app/` subdirectory of the repository, that means the new shield directory should be: ```bash @@ -66,7 +75,11 @@ that make sense to have different defaults when this shield is used. One main it that usually has a new default value set here is the `ZMK_KEYBOARD_NAME` value, which controls the display name of the device over USB and BLE. -The updated new default values should always be wrapped inside a conditional on the shield config name defined in the `Kconfig.shield` file. Here's the simplest example file: +The updated new default values should always be wrapped inside a conditional on the shield config name defined in the `Kconfig.shield` file. Here's the simplest example file. + +:::warning +Do not make the keyboard name too long, otherwise the bluetooth advertising might fail and you will not be able to find your keyboard from your laptop / tablet. +::: ``` if SHIELD_MY_BOARD diff --git a/docs/docs/dev-guide-usb-logging.md b/docs/docs/dev-guide-usb-logging.md index d0274d1..50884be 100644 --- a/docs/docs/dev-guide-usb-logging.md +++ b/docs/docs/dev-guide-usb-logging.md @@ -12,11 +12,25 @@ If you are developing ZMK on a device that does not have a built in UART for deb Zephyr can be configured to create a USB CDC ACM device and the direct all `printk`, console output, and log messages to that device instead. +:::warning Battery Life Impact + +Enabling logging increases the power usage of your keyboard, and can have a non-trivial impact to your time on battery. +It is recommended to only enable logging when needed, and not leaving it on by default. + +::: + ## Kconfig The following KConfig values need to be set, either by copy and pasting into the `app/prj.conf` file, or by running `west build -t menuconfig` and manually enabling the various settings in that UI. +:::note +If you are debugging your own keyboard in your [user config repository](./user-setup.md), use +`config/boards/shields/<your_keyboard>/<your_keyboard>.conf` instead of `app/prj.conf`. In Github +Actions, you can search the `Kconfig file` build log to verify the options above have been enabled +for you successfully. +::: + ``` # Turn on logging, and set ZMK logging to debug output CONFIG_LOG=y @@ -52,6 +66,7 @@ defaultValue="linux" values={[ {label: 'Linux', value: 'linux'}, {label: 'Windows', value: 'win'}, +{label: 'MacOS', value: 'macos'} ]}> <TabItem value="linux"> @@ -73,6 +88,17 @@ On Windows, you can use [PuTTY](https://www.putty.org/). Once installed, use Dev If you already have the Ardunio IDE installed you can also use its built-in Serial Monitor. </TabItem> +<TabItem value="macos"> + +On MacOS, the device name is something like `/dev/tty.usbmodemXXXXX` where `XXXXX` is some numerical ID. +You can connect to the device with [tio](https://tio.github.io/) (can be installed via [Homebrew](https://formulae.brew.sh/formula/tio)): + +``` +sudo tio /dev/tty.usbmodem14401 +``` + +You should see tio printing `Disconnected` or `Connected` when you disconnect or reconnect the USB cable. +</TabItem> </Tabs> From there, you should see the various log messages from ZMK and Zephyr, depending on which systems you have set to what log levels. diff --git a/docs/docs/feature/keymaps.md b/docs/docs/feature/keymaps.md index 56fc2cc..a322336 100644 --- a/docs/docs/feature/keymaps.md +++ b/docs/docs/feature/keymaps.md @@ -68,7 +68,7 @@ In this case, the `A` is actually a define for the raw HID keycode, to make keym For example of a binding that uses two parameters, you can see how "mod-tap" (`mt`) is bound: ``` -&mt MOD_LSFT D +&mt LSFT D ``` Here, the first parameter is the set of modifiers that should be used for the "hold" behavior, and the second @@ -92,7 +92,7 @@ The top two lines of most keymaps should include: The first defines the nodes for all the available behaviors in ZMK, which will be referenced in the behavior bindings. This is how bindings like `&kp` can reference the key press behavior defined with an anchor name of `kp`. -The second include brings in the defines for all the keycodes (e.g. `A`, `NUM_1`, `M_PLAY`) and the modifiers (e.g. `MOD_LSFT`) used for various behavior bindings. +The second include brings in the defines for all the keycodes (e.g. `A`, `NUM_1`, `M_PLAY`) and the modifiers (e.g. `LSFT`) used for various behavior bindings. ### Root devicetree Node diff --git a/docs/docs/intro.md b/docs/docs/intro.md index 547265a..cb227a2 100644 --- a/docs/docs/intro.md +++ b/docs/docs/intro.md @@ -16,7 +16,7 @@ ZMK is currently missing some features found in other popular firmware. This tab | ---------------------------------------------------------------------------------------------------------------------- | :-: | :-------: | :-: | | Low Latency BLE Support | ✅ | ✅ | | | Multi-Device BLE Support | ✅ | | | -| USB Connectivity | ✅ | | ✅ | +| [USB Connectivity](behavior/outputs) | ✅ | | ✅ | | User Configuration Repositories | ✅ | | | | Split Keyboard Support | ✅ | ✅ | ✅ | | [Keymaps and Layers](behavior/layers) | ✅ | ✅ | ✅ | diff --git a/docs/docs/user-setup.md b/docs/docs/user-setup.md index 5a4921e..990a8f6 100644 --- a/docs/docs/user-setup.md +++ b/docs/docs/user-setup.md @@ -39,6 +39,7 @@ The remainder of this guide assumes the following prerequisites: 1. You have an active, working [GitHub](https://github.com/) account. 1. You have installed and configured the [`git`](https://git-scm.com/) version control tool. +1. You have locally configured git to access your github account. If using [personal access tokens](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token), please be sure it was created with the "workflow" scope option selected. :::note If you need to, a quick read of [Learn The Basics Of Git In Under 10 Minutes](https://www.freecodecamp.org/news/learn-the-basics-of-git-in-under-10-minutes-da548267cc91/) will help you get started. @@ -102,6 +103,12 @@ Pick an MCU board: ### Keyboard Shield Selection +:::note +If you are building firmware for a new keyboard shield that is not included in the built-in +list of shields, you can choose any shield from the list that is similar to yours to generate the repository, +and edit / add necessary files according to the [guide for adding new keyboard shield](./dev-guide-new-shield.md). +::: + When prompted, enter the number for the corresponding keyboard shield you would like to target: ``` diff --git a/docs/package.json b/docs/package.json index f1e911e..cb42506 100644 --- a/docs/package.json +++ b/docs/package.json @@ -8,7 +8,10 @@ "serve": "docusaurus serve", "swizzle": "docusaurus swizzle", "deploy": "docusaurus deploy", - "clear": "docusaurus clear" + "clear": "docusaurus clear", + "lint": "eslint . --ext js,jsx,md,mdx", + "prettier:check": "prettier --check .", + "prettier:format": "prettier --write ." }, "dependencies": { "@docusaurus/core": "^2.0.0-alpha.66", diff --git a/docs/sidebars.js b/docs/sidebars.js index 54f6576..c8dc79f 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -22,6 +22,7 @@ module.exports = { "behavior/mod-tap", "behavior/reset", "behavior/bluetooth", + "behavior/outputs", "behavior/lighting", "behavior/power", ], |