Commit Graph

15474 Commits

Author SHA1 Message Date
Trent Piepho 0d93392f10 py/mkrules.mk: List hash files as byproducts.
These are produced by the "cat" command to makeqstrdefs.py, to allow it to
not update unchanged files.  cmake doesn't know about them and so they are
not removed on a "clean".

This triggered a bug in makeqstrdefs.py where it would not recreate a
deleted output file (which is removed by clean) if a stale hash file with a
valid hash still existed.

Listing them as byproducts will cause them to be deleted on clean.

Signed-off-by: Trent Piepho <tpiepho@gmail.com>
2023-12-15 15:48:07 +11:00
stijn 85c02166ca py/modsys: Implement optional sys.intern.
Signed-off-by: stijn <stijn@ignitron.net>
2023-12-15 11:43:39 +11:00
Carlosgg 05d3b22301 docs/library: Document SSLContext cert methods and asyncio support.
Add `load_cert_chain`, `load_verify_locations`, `get_ciphers` and
`set_ciphers` SSLContext methods in ssl library, and update asyncio
`open_connection` and `start_server` methods with ssl support.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-14 13:06:39 +11:00
Carlosgg bfd6ad94ff extmod/asyncio: Add ssl support with SSLContext.
This adds asyncio ssl support with SSLContext and the corresponding
tests in `tests/net_inet` and `tests/multi_net`.

Note that not doing the handshake on connect will delegate the handshake to
the following `mbedtls_ssl_read/write` calls.  However if the handshake
fails when a client certificate is required and not presented by the peer,
it needs to be notified of this handshake error (otherwise it will hang
until timeout if any).  Finally at MicroPython side raise the proper
mbedtls error code and message.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-14 12:20:19 +11:00
Damien George f33dfb966a extmod/modssl_mbedtls: Fix parsing of ciphers in set_ciphers method.
Fixes two issues:
- None should not be allowed in the list, otherwise the corresponding entry
  in ciphersuites[i] will have an undefined value.
- The terminating 0 needs to be put in ciphersuites[len].

Signed-off-by: Damien George <damien@micropython.org>
2023-12-14 12:03:36 +11:00
Damien George bba8a673d5 tests: Update SSL network tests to use SSLContext, and work on CPython.
Changes are:
- use ssl.SSLContext.wrap_socket instead of ssl.wrap_socket
- disable check_hostname and call load_default_certs() where appropriate,
  to get CPython to run the tests correctly
- pass socket.AF_INET to getaddrinfo and socket.socket(), to force IPv4
- change tests to use github.com instead of google.com, because certificate
  validation was failing with google.com

Signed-off-by: Damien George <damien@micropython.org>
2023-12-12 21:22:10 +11:00
Damien George ef996d15b9 extmod/modssl_mbedtls: Make SSLSocket.getpeercert() optional.
And only enable this method when the relevant feature is available in
mbedtls.  Otherwise, if mbedtls doesn't support getting the peer
certificate, this method always returns None and it's confusing why it does
that.  It's better to remove the method altogether, so the error trying to
use it is more obvious.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-12 21:21:54 +11:00
Damien George c9eb6bc601 esp32: Re-enable custom mbedtls error string tables.
To match other ports that use mbedtls.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-12 16:25:07 +11:00
Damien George 58e75264c1 lib/mbedtls_errors: Update error list for latest esp32 mbedtls.
Running `./do-esp32.sh` now generates this esp32_mbedtls_errors.c file,
with IDF v5.0.4.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-12 16:25:07 +11:00
Carlosgg f3f215e9bd extmod/modssl_mbedtls: Add SSLContext certificate methods.
This commit adds:

1) Methods to SSLContext class that match CPython signature:

	- `SSLContext.load_cert_chain(certfile, keyfile)`
	- `SSLContext.load_verify_locations(cafile=, cadata=)`
	- `SSLContext.get_ciphers()` --> ["CIPHERSUITE"]
	- `SSLContext.set_ciphers(["CIPHERSUITE"])`

2) `sslsocket.cipher()` to get current ciphersuite and protocol
   version.

3) `ssl.MBEDTLS_VERSION` string constant.

4) Certificate verification errors info instead of
   `MBEDTLS_ERR_X509_CERT_VERIFY_FAILED`.

5) Tests in `net_inet` and `multi_net` to test these new methods.

`SSLContext.load_cert_chain` method allows loading key and cert from disk
passing a filepath in `certfile` or `keyfile` options.

`SSLContext.load_verify_locations`'s `cafile` option enables the same
functionality for ca files.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-12 16:25:07 +11:00
IhorNehrutsa 4365edb810 esp32: Change minimum supported IDF version to v5.0.4.
Also, IDF v5.1.2 is now supported, just not used by default.

IDF v5.0.2 still builds but we cannot guarantee continued support for this
version moving forward.

Signed-off-by: IhorNehrutsa <IhorNehrutsa@gmail.com>
2023-12-11 13:00:09 +11:00
Damien George d0758d8a33 esp32/boards/ESP32_GENERIC: Disable network.LAN and VM-opt on D2WD.
To reduce firmware size, because IDF v5.0.4 has increased in size.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-11 13:00:06 +11:00
Damien George b4b77c17b5 esp32/mpconfigport: Keep some funcs out of IRAM for ESP32-SPIRAM builds.
To make sure the build fits.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-11 12:59:51 +11:00
Ihor Nehrutsa 3069fee386 esp32/machine_i2s: Fix deprecated fields and constants.
Signed-off-by: IhorNehrutsa <Ihor.Nehrutsa@gmail.com>
2023-12-11 12:59:29 +11:00
Ihor Nehrutsa a427117d03 esp32/modmachine: Fix deprecated esp_pm_config_XXX_t.
Co-Authored-By: Trent Piepho <35062987+xyzzy42@users.noreply.github.com>

Signed-off-by: IhorNehrutsa <Ihor.Nehrutsa@gmail.com>
2023-12-11 12:59:14 +11:00
Ihor Nehrutsa e423b3c0ba docs/esp32/quickref: Add DAC example.
Signed-off-by: IhorNehrutsa <Ihor.Nehrutsa@gmail.com>
2023-12-11 12:58:22 +11:00
IhorNehrutsa 495be71d56 esp32/machine_dac: Support one-shot mode of driver.
And simplify board configuration of DAC by using SOC_DAC_SUPPORTED.

Signed-off-by: IhorNehrutsa <Ihor.Nehrutsa@gmail.com>
2023-12-11 12:57:43 +11:00
IhorNehrutsa 3106ee4885 esp32/modnetwork: Add WiFi AUTH_WPA3_ENT_192 authenticate mode.
Available in newer versions of the IDF.

Signed-off-by: IhorNehrutsa <IhorNehrutsa@gmail.com>
2023-12-11 12:57:05 +11:00
IhorNehrutsa 88778be73a esp32/machine_i2c: Use APB_CLK_FREQ instead of I2C_APB_CLK_FREQ.
Signed-off-by: IhorNehrutsa <Ihor.Nehrutsa@gmail.com>
2023-12-11 12:56:38 +11:00
Carlosgg c393cd7e48 esp32/boards/sdkconfig.base: Disable unused mbedtls options.
Disable unused EC curves and default certificate bundle which is not
implemented in MicroPython.  This reduces the firmware size significantly.

This follows commit 68f166dae9.

Signed-off-by: Carlos Gil Gonzalez <carlosgilglez@gmail.com>
2023-12-11 12:06:35 +11:00
Damien George e1a7aa23fd ports: Switch build to use common lib/libm list of source files.
Signed-off-by: Damien George <damien@micropython.org>
2023-12-08 15:42:41 +11:00
Damien George 241dbac928 extmod: Add lists of libm/libm_dbl source files for ports to use.
So that ports don't need to specify each of these files, they can simply
refer to the appropriate make/cmake variable.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-08 15:39:28 +11:00
Angus Gratton 2c828a8815 unix: Update port to use the new event functions.
Necessary to get coverage of the new event functions.

Deletes the case that called usleep(delay) for mp_hal_delay_ms(), it seems
like this wouldn't have ever happened anyhow (MICROPY_EVENT_POOL_HOOK is
always defined for the unix port).

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 13:17:15 +11:00
Angus Gratton 73879734d9 esp8266: Update port to use new event functions.
This is necessary to avoid watchdog timeout in long i2c.scan(), as
previously machine_i2c.c would call MICROPY_EVENT_POLL_HOOK if
MICROPY_EVENT_POLL_HOOK_FAST was not available.

Compared to previous implementation, this implementation removes the
ets_event_poll() function and calls the SDK function ets_loop_iter() from
MICROPY_INTERNAL_EVENT_HOOK instead.  This allows using the port-agnostic
functions in more places.

There is a small behaviour change, which is that the event loop gets
iterated in a few more places (i.e. anywhere that mp_event_handle_nowait()
is called).  However, this looks like maybe only modselect.c - and is
probably good to process Wi-Fi events in that polling loop.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 12:50:38 +11:00
Angus Gratton c393f5c123 windows: Implement MICROPY_INTERNAL_WFE() macro.
This should be the equivalent of the previous event poll hook macro.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 12:49:54 +11:00
Angus Gratton 393938b3e6 rp2/main: Enable SEVONPEND CPU interrupt bit.
Previously this was not set, so potential for race conditions in interrupt
handlers this didn't issue SEV.  (Which is currently all of them, as far as
I can see.)

Eventually we might be able to augment the interrupt handlers that wake the
main thread to call SEV, and leave the others as-is to suspend the CPU
slightly faster, but this will solve the issue for now.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 12:49:49 +11:00
Angus Gratton 28529351ae rp2: Switch rp2 and drivers to use new event functions.
This commit changes all uses in the rp2 port, and drivers that are
optionally supported by that port.

The old MICROPY_EVENT_POLL_HOOK and MICROPY_EVENT_POLL_HOOK_FAST macros are
no longer used for rp2 builds and are removed (C user code will need to be
changed to suit).

Also take the opportunity to change some timeouts that used 64-bit
arithmetic to 32-bit, to hopefully claw back a little code size.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 12:49:43 +11:00
Angus Gratton df3948d3c2 extmod: Switch to use new event functions.
See previous commit for details of these functions.  As of this commit,
these still call the old hook macros on all ports.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 12:48:50 +11:00
Angus Gratton f5be0128e4 py: Add port-agnostic inline functions for event handling.
These are intended to replace MICROPY_EVENT_POLL_HOOK and
MICROPY_EVENT_POLL_HOOK_FAST, which are insufficient for tickless ports.

This implementation is along the lines suggested here:
https://github.com/micropython/micropython/issues/12925#issuecomment-1803038430

Currently any usage of these functions expands to use the existing hook
macros, but this can be switched over port by port.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-08 12:47:00 +11:00
Angus Gratton 66be82da7c esp8266: Avoid including ep_mphal.h directly.
This header has no include guards and is apparently only supposed to be
included from py/mphal.h.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-07 14:07:21 +11:00
iabdalkader 0960d64d3b extmod/network_ninaw10: Switch to using soft-timer for polling.
This patch simplifies the connection and sockets polling code, by switching
to a soft-timer to schedule polling code, and by using one node for
scheduling.  This also fixes any issues that could result from using a heap
allocated machine_timer, and includes better handling of the sockets poll
list.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2023-12-06 17:32:21 +11:00
iabdalkader 50f31cc902 extmod/modnetwork: Add deinit function to NIC protocol.
This is usually called on soft-reboot, a NIC can implement this to do any
necessary cleaning up (such as invalidating root pointers).

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2023-12-06 17:32:00 +11:00
Damien George d30d5c99af tests/run-tests.py: Skip Thumb2 tests if target doesn't support them.
Signed-off-by: Damien George <damien@micropython.org>
2023-12-06 16:05:37 +11:00
Damien George b796f1e3f1 tests/float/inf_nan_arith.py: Include -inf in argument combos.
This adds tests for, eg, -inf + inf which should be nan.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-06 14:06:01 +11:00
Damien George 3f2c423686 rp2: Switch to locally provided math library.
This commit fixes all known floating-point bugs with the pico-sdk.  There
are two things going on here:
- Use a custom pico float component so that the pico-sdk doesn't include
  its math functions, and then provide our own from lib/libm.
- Provide a wrapper for __aeabi_fadd to fix the infinity addition bug.

Prior to this commit, the following tests failed on the rp2 port: cmath_fun
float_parse math_domain math_domain_special math_fun_special.  With this
commit, all these tests pass.

Thanks to @projectgus for how to approach this fix.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-06 14:00:07 +11:00
Scott Zhao 1323a71823 esp32/boards/UM_TINYPICO: Fix typo in baudrate instructions.
The installation instructions for ESP32 TinyPICO board contained a typo
that used a non-standard baud rate 912600 instead of 921600.  This made the
upload command fail on some Windows computers.

Signed-off-by: Scott Zhao <zhaomh1998@outlook.com>
2023-12-06 13:46:04 +11:00
Thomas Wenrich 78b3fe5689 esp32/machine_rtc: Preserve RTC user memory over most reset causes.
The user memory area - accessible by machine.RTC.memory() -- will now
survive most reboot causes.  A power-on reset (also caused by the EN pin on
some boards) will clean the memory.  When this happens, the magic number
not found in the user memory will cause initialization.

After other resets (triggered by watchdogs, machine.reset(), ...), the user
is responsible to check and validate the contents of the user area.

This new behaviour can be changed by enabling
MICROPY_HW_RTC_MEM_INIT_ALWAYS: in that case the RTC memory is always
cleared on boot.

Signed-off-by: Thomas Wenrich <twenrich@gmail.com>
2023-12-06 13:31:13 +11:00
Damien George d3595fed2b tools/ci.sh: Build ESP32_GENERIC-SPIRAM as part of esp32 CI.
Signed-off-by: Damien George <damien@micropython.org>
2023-12-06 12:37:25 +11:00
Damien George 69089a532e esp32/boards: Enable further IRAM saving opts to fit ESP32-SPIRAM fw.
The amount of free IRAM in ESP32 SPIRAM builds is very small and went over
the limit due to commit 30b0ee34d3.  This
commit enables further optimisations to reduce IRAM usage.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-06 12:35:45 +11:00
Angus Gratton 527c4b0497 github/workflows: Enable ccache for esp32 build.
CCaches are scoped per-job.

Uses https://github.com/hendrikmuhs/ccache-action to get desired behaviour
(updating the cache on each run).

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-06 12:15:33 +11:00
Angus Gratton 731a1f5233 github/workflows: Enable build matrix for stm32 port.
This is for consistency with the previous commit that uses this approach
for esp32.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-06 12:15:20 +11:00
Angus Gratton b6df8f8452 github/workflows: Use build matrix for esp32 port.
Allows splitting the esp32 job into multiple parts without too much
boilerplate.  The matrix is parameterised using the name of the function to
call in tools/ci.sh, to minimise the dependency on GitHub Actions.

This can get esp32 build times down around 3m if IDF is cached already.

If the cache is cold, the cache preparation step on each job can double up
against each other.  However, restructuring the workflow to not do this
seems either complex or requires copy-pasting the entire cache step.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-06 12:13:56 +11:00
Angus Gratton 9f620d2819 github/workflows: Cache ESP-IDF checkout and installation.
Cache is keyed on the ESP-IDF version used in CI, so there shouldn't be any
cache invalidation issues when ESP-IDF version changes.

Restoring from cache takes approx 15s, compared to 2-3m to perform these
steps (ESP-IDF tools install, ESP-IDF clone, ESP-IDF submodule clone) the
first time.

Cache size is approx 1.6GB, the git clone is tweaked as much as possible to
keep the size down.

Signed-off-by: Angus Gratton <angus@redyak.com.au>
2023-12-06 12:12:52 +11:00
MikeTeachman 58112fc49a stm32/machine_i2s: Improve accuracy of SCK frequency.
Configures the I2S PLL to produce a frequency that the I2S clock generator
can use to create an optimal SCK frequency.  The I2S PLL configuration
table is automatically generated at build time.

Fixes issue #10280.

Signed-off-by: Mike Teachman <mike.teachman@gmail.com>
2023-12-06 11:48:03 +11:00
Damien George f3889db265 stm32/usbd_cdc_interface: Include header to get machine_bootloader decl.
This was broken by commit 48b5a7b060.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-04 22:19:59 +11:00
Damien George 20af857f52 nrf/boards/ARDUINO_NANO_33_BLE_SENSE: Don't enable MICROPY_MBFS.
This board has MICROPY_VFS enabled, which should take precedence over
MICROPY_MBFS (and did prior to 22d9116c8c).

Signed-off-by: Damien George <damien@micropython.org>
2023-12-04 22:18:14 +11:00
Damien George d422b68d10 nrf/mpconfigport: Enable MICROPY_PY_MACHINE_BOOTLOADER.
This should have been enabled in 48b5a7b060
but was missed.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-04 22:17:18 +11:00
Carlosgg 30b0ee34d3 esp32: Enable mbedtls cert time validation.
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-04 10:50:42 +11:00
Damien George b5449b0f09 extmod/mbedtls: Enable certificate time/date validation by default.
All ports using this common configuration already enable time/date
validation, so this commit is a no-op change.

Signed-off-by: Damien George <damien@micropython.org>
2023-12-01 15:08:11 +11:00
Carlosgg 20ffbe1a41 unix/mbedtls: Enable mbedtls cert time validation.
To match other ports.

Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
2023-12-01 15:08:11 +11:00