Commit Graph

627 Commits

Author SHA1 Message Date
Andrew Leech d2a9d70c09 tools/ci.sh: Ensure enough commits are fetched for a common ancestor.
This commit updates the ci script to automatically fetch all upstream if
the common commit hasn't been found; this should preserve the speed of CI
checks for most PR's, and use a reliable but slow fetch if needed for older
ones.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2023-10-11 14:05:14 +11:00
Jim Mussared 69e34b6b6b all: Switch to new preview build versioning scheme.
See https://github.com/micropython/micropython/issues/12127 for details.

Previously at the point when a release is made, we update mpconfig.h
and set a git tag. i.e. the version increments at the release.

Now the version increments immediately after the release. The workflow is:
1. Final commit in the cycle updates mpconfig.h to set (X, Y, 0, 0) (i.e.
   clear the pre-release state).
2. This commit is tagged "vX.Y.0".
3. First commit for the new cycle updates mpconfig.h to set (X, Y+1, 0, 1)
   (i.e. increment the minor version, set the pre-release state).
4. This commit is tagged "vX.Y+1.0-preview".

The idea is that a nightly build is actually a "preview" of the _next_
release. i.e. any documentation describing the current release may not
actually match the nightly build. So we use "preview" as our semver
pre-release identifier.

Changes in this commit:
 - Add MICROPY_VERSION_PRERELEASE to mpconfig.h to allow indicating that
   this is not a release version.
 - Remove unused MICROPY_VERSION integer.
 - Append "-preview" to MICROPY_VERSION_STRING when the pre-release state
   is set.
 - Update py/makeversionhdr.py to no longer generate MICROPY_GIT_HASH.
 - Remove the one place MICROPY_GIT_HASH was used (it can use
   MICROPY_GIT_TAG instead).
 - Update py/makeversionhdr.py to also understand
   MICROPY_VERSION_PRERELEASE in mpconfig.h.
 - Update py/makeversionhdr.py to convert the git-describe output into
   semver-compatible "X.Y.Z-preview.N.gHASH".
 - Update autobuild.sh to generate filenames using the new scheme.
 - Update remove_old_firmware.py to match new scheme.
 - Update mpremote's pyproject.toml to handle the "-preview" suffix in the
   tag. setuptools_scm maps to this "rc0" to match PEP440.
 - Fix docs heading where it incorrectly said "vvX.Y.Z" for release docs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-10-06 12:10:14 +11:00
Damien George 040a96d00e docs,tools: Change remaining "urequests" references to "requests".
Signed-off-by: Damien George <damien@micropython.org>
2023-10-05 14:04:45 +11:00
Damien George 92717a95c0 tools/metrics.py: Fix esp32 and esp8266 board names after renaming.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-04 15:44:25 +11:00
Damien George cf490a7091 all: Fix various spelling mistakes found by codespell 2.2.6.
Signed-off-by: Damien George <damien@micropython.org>
2023-10-03 11:24:50 +11:00
Jos Verlinde 58c2c503a9 tools/mpremote: Add support for rfc2217, serial over TCP.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-09-29 14:40:17 +10:00
Jim Mussared 88ecc78eb3 tools/autobuild/build-downloads.py: Verify standard features.
Defines the list of standard features and ensures that each board.json
only uses those ones. This list can be extended, but needs to be a
deliberate decision.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-29 14:18:33 +10:00
Damien George b0e03b3e07 tools/autobuild: Include .bin firmware in renesas-ra build output.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-18 11:05:14 +10:00
iabdalkader 174bb28d8e tools/ci.sh: Add ARDUINO_PORTENTA_C33 to RA CI build.
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2023-09-15 00:10:52 +10:00
Damien George 6aa404ca53 tools/metrics.py: Fix nrf and rp2 board names after renaming.
Signed-off-by: Damien George <damien@micropython.org>
2023-09-02 00:37:45 +10:00
Jim Mussared 607548f32d examples/natmod: Add features4 as a class definition example.
Also provide a basic README.md for dynamic native modules.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-02 00:16:16 +10:00
Jim Mussared a64f2fdca0 py/dynruntime.h: Implement MP_OBJ_NEW_QSTR.
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).

Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).

Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.

Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-09-01 16:14:22 +10:00
Jon Nordby 4837ec336a tools/mpy_ld.py: Support more complex rodata sections.
Sections sometimes named .rodata.str1.1 etc, instead of just .rodata.
Avoid crashing in that case.  Instead treat it like any other RO section.

Fix thanks to @phlash.

Fixes issue #8783.

Signed-off-by: Jon Nordby <jononor@gmail.com>
2023-09-01 15:30:21 +10:00
Jim Mussared bc1af5ab7f cc3200/Makefile: Build firmware.zip.
This allows the cc3200 port to be build with the standard autobuild script
rather than the custom build-cc3200-latest.sh (which is now removed).

This also fixes the path inside the zip file (by using the `-j` flag to
zip).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-24 15:44:20 +10:00
Jim Mussared 873bd00ad4 rp2: Rename PICO, PICO_W to RPI_PICO, RPI_PICO_W.
PICO might not always be a unique name across all ports, and the
convention generally for other boards is to do VENDOR_BOARD.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 15:49:37 +10:00
Jim Mussared ef03ca8bf2 esp8266: Add board variant support.
This merges the existing GENERIC, GENERIC_1M, and GENERIC_512k boards
into variants of the new ESP8266_GENERIC board (renamed from GENERIC so
as not to clash with other ports).

Also moves the generation of the "OTA" variant (previously generated by
autobuild/build-esp8266-latest.sh) into the variant.

Following the convention established for the WEACTSTUDIO rp2 board, the
names of the variants are FLASH_1M and FLASH_512K (but rename the .ld files
to use MiB and kiB).

Updates autobuild to build esp8266 firmware the same way as other ports.
This requires renaming the output from firmware-combined.bin to just
firmware.bin.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 15:49:37 +10:00
Jim Mussared aa23698119 esp32: Rename GENERIC* boards to ESP32_GENERIC*.
Board names need to be unique across ports, and GENERIC clashes with
the ESP8266 (which will be renamed to ESP8266_GENERIC).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 13:54:37 +10:00
Jim Mussared b5836e7252 esp32: Use uppercase variant names.
This is to support a future change to add the variant name to the build
directory and therefore should be the same style as the board name.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 13:54:37 +10:00
Jim Mussared f007d0aa3d nrf/boards: Rename all nRF boards to use uppercase.
This is to match all the other ports.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-23 13:54:37 +10:00
Angus Gratton 1a5c9b9da4 tools/mpy-tool.py: Ignore linter failure in Python 2 compatibility code.
Found by Ruff checking F821.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-16 16:16:44 +10:00
Angus Gratton b8189d039d tools/mpy_ld.py: Pre-declare some local variables to appease linter.
Spurious fix as the logic is structured such that these variables will be
set before dereferenced, but this keeps Ruff happy (no more F821
undefined-name).

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-16 16:16:44 +10:00
Jim Mussared 91674c41b8 tools/autobuild: Automatically build all variants for each board.
Removes the special-case for stm32.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-15 17:38:02 +10:00
Jim Mussared 4bbe879c26 tools/mpremote: Make soft-reset count as an action.
Otherwise `mpremote soft-reset` will implicitly run the repl command.

Fixes issue #10871.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-08-10 14:11:53 +10:00
Jos Verlinde 232c29c2ed tools/codeformat.py: Use pyproject.toml for black config.
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2023-08-10 11:54:36 +10:00
Daniël van de Giessen 8dbb29da3f tools/codeformat.py: Skip formatting ESP-IDF managed components.
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2023-08-10 11:36:24 +10:00
Angus Gratton 597fcb4751 tools/mpy-tool.py: Use isinstance() for type checking.
Ruff version 283 expanded E721 to fail when making direct comparison
against a built-in type.  Change the code to use isinstance() as
suggested, these usages appear to have equivalent functionality.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-08-09 17:41:54 +10:00
Damien George d14ddcbdb5 tools/autobuild: Add support for application .bin files for esp32.
On esp32, the build output consists of:
- micropython.elf
- micropython.map
- micropython.bin -- application only
- micropython.uf2 -- application only
- firmware.bin -- bootloader, partition table and application

Currently everything is available at the download page except
micropython.bin.  This commit adds that file but with the extension changed
to .app-bin, to distinguish it from .bin (the full thing).

Signed-off-by: Damien George <damien@micropython.org>
2023-07-27 13:10:19 +10:00
Jim Mussared ea1a5e43d0 examples/natmod/deflate: Add deflate as a dynamic native module.
This replaces the previous zlib version.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 19:32:42 +10:00
Jim Mussared add1200343 all: Remove the zlib module.
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-07-21 18:48:29 +10:00
Damien George 273ba28c07 tools/autobuild: Update auto-build code to build esp32 port with IDF v5.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:53:04 +10:00
Damien George 6a9db521ed github/workflows: Update esp32 CI to use IDF v5.0.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-23 15:34:53 +10:00
Damien George 0908d00453 tools/ci.sh: Add a H5 board to stm32 CI build.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-15 11:09:37 +10:00
Damien George 33b403dfb4 tools/ci.sh: Build PICO_W board as part of rp2 CI.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-14 22:20:20 +10:00
Wilko Nienhaus 034502bc72 esp32/esp32_ulp: Fix ULP (FSM) support for S2 and S3.
This change enables the ULP (FSM) for all ESP32 variants rather than
requiring it to be enabled for each board specifically.

It also ensures the correct header file is included for each variant.

Lastly, it updates the IDF version we're builing against to v4.4.2, as that
version contains important fixes to make the ULP actually work on S2/S3
chips. See: https://github.com/espressif/esp-idf/commit/a0e3d48

Signed-off-by: Wilko Nienhaus <wilko.nienhaus@gmail.com>
2023-06-14 16:41:03 +10:00
Jim Mussared b4de697ad1 tools/mpremote: Fix exec_ -> exec in commands.py.
This was missed in the pyboard refactor and is preventing `cp -r` from
working.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-09 13:36:12 +10:00
Damien George 0832fc53ce tools/ci.sh: Add mimxrt and samd ports to code size build.
The automatic code size build and GitHub comment is a really useful
feature.  This commit adds a few more builds to it (mimxrt and samd).

Signed-off-by: Damien George <damien@micropython.org>
2023-06-08 22:47:03 +10:00
Jim Mussared a1fbb1980c extmod/modtimeq: Remove timeq module.
This is a MicroPython-specific module that existed to support the old
version of uasyncio.  It's undocumented and not enabled on all ports and
takes up code size unnecessarily.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:28 +10:00
Jim Mussared 5fd042e7d1 all: Replace all uses of umodule in Python code.
Applies to drivers/examples/extmod/port-modules/tools.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 4216bc7d13 tests: Replace umodule with module everywhere.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared eb85f4d4c9 examples/natmod: Rename umodule to module.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:24 +10:00
Jim Mussared 45ac651d1a all: Rename *umodule*.c to remove the "u" prefix.
Updates any includes, and references from Makefiles/CMake.

This essentially reverts what was done long ago in commit
136b5cbd76

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:17 +10:00
Jim Mussared f5f9edf645 all: Rename UMODULE to MODULE in preprocessor/Makefile vars.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-08 17:54:11 +10:00
Damien George a61d40e5e8 tools/mpremote: Fix use of stdout_write_bytes function.
Signed-off-by: Damien George <damien@micropython.org>
2023-06-02 18:03:16 +10:00
Jim Mussared b4d785fa20 tools/mpremote: Detach mpremote from pyboard.py.
This commit just takes the necessary parts of pyboard.py and merges them
with pyboardextended.py to make a new transport_serial.py, and updates the
rest of mpremote to use this instead.

It is difficult to continue to add features to mpremote (which usually
requires modification to pyboard.py) while also maintaining backwards
compatibility for pyboard.py.

The idea is that this provides a starting point for further refactoring of
mpremote to allow different transports (webrepl, BLE, etc).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 17:42:13 +10:00
Jim Mussared bd5d0163c4 docs/reference/mpremote.rst: Extend the mpremote guide.
Changes in this commit:
- Add a extra detail to each of the commands.
- Add more about handling options and arguments.
- Include shortcut commands that behave like real commands to the command
  list (e.g. bootloader, rtc).
- Add extra information and reword to address common misconceptions, in
  particular how commands chain together.
- Add additional examples showing some more interesting combinations.
- Add descriptions to each of the examples.
- Add pipx installation instructions.
- Describe how user-configuration works.

This work was sponsored by Google Season of Docs.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:28 +10:00
Jim Mussared b6b19798c2 tools/mpremote: Handle `cp` without destination.
This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:16:24 +10:00
Jim Mussared 46715e370d tools/mpremote: Add `rtc` commands to get and set the RTC.
Replaces the existing functionality provided by the `setrtc` alias to use
the current time, rather than a hard-coded date/time.

Use `rtc` to query the current time.  Use `rtc --set` to set it.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:15:48 +10:00
Jim Mussared d736a2f3f3 tools/mpremote: Allow terminator for shortcut commands.
For example, the `reset` shortcut previously allowed an optional delay, but
the argument handling cannot handle `reset next-command` as `next-command`
will be interpreted as the delay argument.  The fix in this commit allows
`reset + next-command`.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:14:26 +10:00
Jim Mussared 7c2c9ea21c tools/mpremote: Add `sleep` command.
This allows the sequence to be paused (e.g. wait for device, etc).

Also removes the t_ms arg in reset/bootloader, because these arguments
don't really need to be changed, and keeping them would mean inconsistent
units used for time delays.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2023-06-02 16:11:29 +10:00
Damien George ed7a3b11d9 tools/pydfu.py: Use getattr to retrieve getargspec function.
Since Python 3.11, inspect.getargspec() has been removed.

Signed-off-by: Damien George <damien@micropython.org>
2023-05-24 12:00:32 +10:00