A versatile image rendering and transform library.
Go to file
Lephenixnoir 653de56d70
cmake: install in the fxSDK sysroot
2022-08-21 17:34:54 +02:00
cmake cmake: install in the fxSDK sysroot 2022-08-21 17:34:54 +02:00
src update to gint 2.4.0 2021-04-27 16:24:42 +02:00
testing README update and image 2020-03-10 15:34:49 +01:00
.gitignore add support for quick build/install with GiteaPC 2021-01-25 15:20:12 +01:00
CMakeLists.txt cmake: install in the fxSDK sysroot 2022-08-21 17:34:54 +02:00
README.md update README 2021-01-29 13:33:52 +01:00
giteapc.make build: fix uninstall command for sh-based OSes 2021-01-29 19:05:32 +01:00
libimg.h add version and find_package() module 2021-01-28 22:47:45 +01:00

README.md

libimg: Image transform and blending for gint

This library contains a variety of functions to perform blending, geometric and color transforms on images, namely:

  • Extract references to subimages
  • Blending with alpha (only full opaque and full transparent)
  • Horizontal and vertical flip (mirror)
  • Rotation by 90, 180 and 270 degrees
  • Upscaling by integer factors
  • Fast brightening and darkening
  • Some visual effects that look cool and are cheap to compute.

(Sample image showcasing the capabilities of libimg.)

This library has been developed for gint, though only two VRAM-related functions are specific to gint. I'm willing to help port it to libfxcg if anyone wants to do that.

Differences between img_t and bopti_image_t

The type of images in this library is called img_t. It is not the same as bopti_image_t.

  • bopti_image_t objects are used by a gint component called bopti which specializes in maximizing rendering performance. They are very fast, especially on fx-9860G, but not flexible.
  • img_t objects are part of libimg. They're not rendered as fast, but they are more versatile and can be transformed in many ways.

There is no function to convert between bopti_image_t and img_t. Both types of images are generated by fxconv at compile time. In the unlikely event that an image needs to be available in both formats, the proper way is to convert it twice with different parameters. But normally, only one format is needed:

  • If the image needs to be read or transformed, use libimg. Set the fxconv parameters of the resource to type:libimg-image and declare it as extern img_t myimage.
  • Otherwise, bopti should be used for maximum performance. Set the fxconv parameters to type:bopti_image and declare it as extern bopti_image_t myimage.

Installing and using libimg in a program

Installing with GiteaPC

libimg can be installed with GiteaPC, a tool designed to automate library maintenance in the fxSDK.

% giteapc install Lephenixnoir/libimg

Installing manually

libimg should be built with the fxSDK, which provides the tools and settings needed to compile for the calculator.

% fxsdk build-fx install
% fxsdk build-cg install

Additionally, this repository has a simple testing system for Linux in the testing folder.

  • Use make from that folder to compile main.c into a libimg executable that reads RGB565 bitmap files into img_t images and performs a few transforms.
  • Use make all to also run it to generate the transform.png file that is shown at the top of this README (uses ImageMagick's convert to get the PNG).

Using in a CMake-based add-in

Find the LibImg package, then link against LibImg::LibImg. You should request the same version for gint and libimg.

find_package(LibImg 2.1 REQUIRED)
target_link_libraries(<target_name> LibImg::LibImg)

Using in a Makefile-based add-in

Link with -limg-fx on fx-9860G and -limg-cg on fx-CG 50. In project.cfg:

LIBS_FX := -limg-fx
LIBS_CG := -limg-cg

Converting images and rendering to VRAM

The first step to use libimg in a calculator program is to convert assets to the img_t format. This is done by setting the type parameter of the resource to libimg-image. When using fxconv-metadata.txt, this would look like this:

sprite.png:
  type: libimg-image
  name: sprite
  # etc

Then the newly-converted image can be accessed from a C program by extern-declaring it as any other fxconv resource. It is ready to use immediately. The definition of img_t and the libimg functions are provided by <libimg.h>, which needs to be included.

#include <libimg.h>
extern img_t const sprite;

Note that converted images are read-only, as is everything generated by fxconv. This means that the declared image cannot be written to or transformed in-place. But more on that later. To remember this read-only status, I added the const keyword; this is purely optional.

To render this shiny image to VRAM, use img_render_vram(). On fx-9860G (Graph 35+E family), this is the only way to render to VRAM because the VRAM has a different format than img_t. On fx-CG 50 (Graph 90+E), more powerful methods are available, which we'll see later.

img_render_vram(sprite, x, y);

img_render_vram(), as every other libimg function, accounts for transparent pixels. Only opaque pixels will be copied to VRAM. This makes it very easy to blend images together by just rendering in sequence.

As a quick note, if you have an image with gray pixels on fx-9860G, you need to use img_render_vram_gray() instead (with the same parameters). When using img_render_vram(), gray pixels will be approximated as black and white.

Creating and destroying new images

Because our converted sprite is read-only, we can't do much more for now. To start working on transforms and cool animations, we need to create new images that can be written to. There are two ways to do this; the first is to create an empty image of a fixed size, and the second is to duplicate an existing image.

img_create() will create a new uninitialized image. Note that the value of the pixels is completely random at this point. We'll see how to fill the whole image with img_fill() later. Sometimes filling is not needed and would only decrease performance, so libimg tries to not get in your way and does not do it automatically.

img_t new_32x48_image = img_create(32, 48);

img_copy() will create a copy of an image. The returned copy will be of the same size as the source, but with duplicated pixels. A copy can always be written to, even if the source is read-only.

img_t copy_of_sprite = img_copy(sprite);

Of course, creating new images can fail. Both functions use malloc() under-the-hood, and malloc() fails if there is not enough memory. In this case, img_create() and img_copy() return a null image, which is like the NULL pointer for images. To see if an image is null, call img_null().

if(img_null(copy_of_sprite)) {
  /* Argh, copy has failed! */
}
else {
  /* Everything is fine, let's go! */
}

A null image does not have pixels, and reading or writing it would be an error. You can use a null image in libimg functions though; they will realize the problem and either do nothing or return another null image.

Also, because there is an malloc(), there has to be a free(). Images created by img_create() and img_copy() need to be destroyed once they are no longer needed. Calling img_destroy() will do just that.

img_destroy(new_32x48_image);
img_destroy(copy_of_sprite);

Note that converted images such as sprite, null images, and subimages (described later) don't need to be destroyed because they are not created with malloc(). To make life easier, img_destroy() will tell them apart and do nothing, so when in doubt just destroy everything after use.

Basic transforms

Let's start playing with these new shiny images. We'll start by flipping the sprite horizontally to simulate walking in the other direction. Let's make a new image of the required size and fill it with white pixels. This is important because our original sprite has transparent pixels. If we don't clear the new image, some of the random pixels will be visible around the flipped sprite.

#include <gint/display.h>

img_t flipped_sprite = img_create(sprite.width, sprite.height);
img_fill(flipped_sprite, C_WHITE);

This example introduces several useful features:

  • The dimension of an image can be accessed by accessing its width and height attributes.
  • The img_fill() function will paint all the pixels of an image in a uniform color.
  • Colors used in libimg are the same as colors used in gint, except for transparency on fx-CG 50 (Graph 90+E), which is 0x0001. More on that soon.

Now we've got a blank image with the same size as sprite, so we can go ahead and flip sprite horizontally into it:

img_hflip(sprite, flipped_sprite);

That's it. Now img_render_vram(flipped_sprite, x, y) will show the flipped version. As long as flipped_sprite is not destroyed, it can be rendered an unlimited amount of times. Note that we were able to transform because flipped_sprite is not read-only. If we tried to flip with sprite as a destination, nothing would happen.

But wait, since flipped_sprite was originally white, this just made a horrible white rectangle on the screen. What we actually wanted is to make the background transparent. We can use the special color IMG_ALPHA to do this.

/* Makes flipped_sprite completely transparent: */
img_fill(flipped_sprite, IMG_ALPHA);

On fx-9860G (Graph 35+E family), libimg colors are exactly the same as gint colors, so IMG_ALPHA is equivalent to C_NONE. On fx-CG 50 (Graph 90+E), things are a bit different. Colors are in the RGB565 format, and all 16-bit values are valid RGB565 colors, so there is no room for a special transparent color. libimg solves this dilemma by deciding that 0x0001 represents transparency. 0x0001 has been chosen because it is an extremely dark color, undistinguishable from black, and it's easy to compare numbers to 1 so it's technically faster than other alternatives. Just remember that IMG_ALPHA should be used to talk about transparency in libimg.

Because filling a new image with transparent pixels is a common operation, a shortcut called img_clear() is provided.

/* Also makes flipped_sprite completely transparent: */
img_clear(flipped_sprite);

So now the code does img_create(), img_clear(), and img_hflip() as a transform. This is also a common sequence, so a shortcut called img_hflip_create() is provided to do exactly that.

img_t flipped_sprite = img_hflip_create(sprite);

All transforms work like this. This includes:

  • img_hflip() and img_vflip(), which flip horizontally and vertically.
  • img_rotate(), which rotates by 0, 90, 180 or 270 degrees.
  • img_upscale() which scales up by integer factors.
  • img_dye() which paints all non-transparent pixels in a single color.
  • img_lighten(), img_whiten() and img_darken() which make colors lighter or darker.

We can already mention a few conventions within the library:

  • All transforms first take the src (source image) and dst (destination image) parameters, then any other parameter of the transform (such as the rotation angle).
  • All transforms have a _create() variant which does not have a dst argument. The destination image is created automatically with just the size of the transformed result and returned.

Subsurfaces and positioning

So far we have only transformed images to destinations of the same size, or in some cases such as rotation and upscaling, of related sizes. Let's say we want to create a full spritesheet with both the original and flipped version of sprite. We have a problem because img_hflip() does not allow us to tell where to write the result in the destination image. In fact, img_hflip() always writes the result in the top-left corner.

This is because libimg has a flexible positioning system that is more powerful than that method. This positioning system is built on the idea of extracting references to subimages.

Let's see an exemple. We'll create a spritesheet of size 32x16 with one 16x16 sprite on the left and one on the right.

img_t spritesheet = img_create(32, 16);

The right sprite is at position (16,0) within spritesheet. If we want to apply a lot of transforms on it, we'll have to keep mentioning these coordinates over and over, which is painful and error-prone. Instead, we can use a function called img_sub() to create a reference to the right half of spritesheet.

img_t right_sprite = img_sub(spritesheet, 16, 0, 16, 16);

img_sub() takes five parameters: the source image, and then the position and size of the subimage of interest, as x, y, w, and h. The right sprite starts at x=16 and y=0, and is of width w=16 and height h=16.

img_sub() does not create a new image. It only gives a new view into the same pixels. Drawing to right_sprite right now will totally draw into the right half of spritesheet! In fact, let's just do this.

img_fill(right_sprite, C_BLACK);

And just like that, we have filled only half of spritesheet. Calling img_fill(spritesheet, C_BLACK) would have filled all of it. See how the positioning system makes img_fill() as versatile as a fill-rectangle function.

With that in mind, we can now create the full spritesheet.

/* Copy the normal sprite */
img_render(sprite, spritesheet);
/* Flip into the right half */
img_hflip(sprite, right_sprite);

Of course, this works with all transforms. Please remember though that the destination image or subimage of a transform needs to be at least as large as the transformed result. If the destination is smaller, the transform function will just do nothing.

The subimage does not have new pixels, so it does not need to be destroyed. img_destroy() can still be called on it; but not destroying it means we don't have to store it in a variable, so we can create the flipped sprite like this:

img_hflip(sprite, img_sub(spritesheet, 16, 0, 16, 16));

This reads as "flip sprite horizontally, and write the result in spritesheet, at position (16,0) in a rectangle of size 16x16".

In this case we know that the size of the rectangle has to be 16x16, because it's the size of the sprite. So we can omit it by setting the width and height to -1. This will create a reference to the subimage that starts at position (16,0) of spritesheet and extends all the way to the bottom right corner. img_hflip() will still only touch the first 16x16 pixels in the top left corner, so it's not a problem if the reference is larger than needed.

img_hflip(sprite, img_sub(spritesheet, 16, 0, -1, -1));

If you feel like this is a common construction, then you're absolutely right! So a shortcut has been made for it. Instead of setting the width and height to -1 we can just use img_at() which will set them for us. img_at(img, x, y) is the same as img_sub(img, x, y, -1, -1). So the transform becomes:

img_hflip(sprite, img_at(spritesheet, 16, 0));

This reads as "flip sprite horizontally and put the result in spritesheet at position (16,0)". It's much shorter than before, and because the reference to subimage does not need to be destroyed, there is no risk of leaking memory.

References to subimages are powerful and can be used in a variety of ways. But there is a golden rule: destroying an image also destroys the subimages. Once we call img_destroy(spritesheet), all subimages including right_sprite become invalid and using them is an error. Always keep an eye on the originals!

As an example of versatility of references to subimages, see how img_sub() can be used on the source image to transform only a part of it!

I have not yet explained what img_render() does, so let's take some time do discuss it.

Direct rendering to VRAM (fx-CG 50)

On fx-9860G, the VRAM does not have the same format as an img_t, so we can't use the VRAM as the destination for a transform. We can only use img_render_vram() to copy an image to VRAM after it has been prepared.

But on fx-CG 50, the VRAM does have the same format as an img_t. Use img_vram() to get a reference to it:

img_t vram_as_an_image = img_vram();

Because it is only a reference, it does not need to be destroyed with img_destroy(). Doing it anyway is a no-operation, as with all other references. When in doubt, destroy everything after use.

This VRAM reference allows to directly transform to VRAM without using temporary images. For instance, we can do:

img_hflip(sprite, img_at(img_vram(), x, y));

Similarly, the following calls both fill a rectangle of the VRAM (although drect() is faster).

#include <gint/display.h>

drect(x, y, w, h, color);
img_fill(img_sub(img_vram(), x, y, w, h), color);

This is where the img_render() function becomes the most useful. This function copies its source to the top-left corner of its destination, without transforming it. Unlike transforms, it clips, so if the destination is smaller than the source, only the visible part will be copied. (Transforms do nothing in this situation.) In fact, on fx-CG 50, img_render_vram() is just a simple form for the following call.

img_render(src, img_at(img_vram(), x, y));

img_render() is used most of the time in this way, to copy (render) images to VRAM. Hence the name.

In-place transforms

Some transforms can operate on a single image without requiring to have a different destination. The image is overwritten during the transform, so the original is lost but no additional meomry is needed. This is called an in-place transform.

In-place transforms are possible only when the size of the input is the same as the size of the output. Currently, this includes everything except rotation of a non-square image by 90 or 270 degrees, and upscaling by a non-trivial factor.

To perform an in-place transform, just use the same source and destination:

img_t sprite_copy = img_copy(sprite);
img_hflip(sprite_copy, sprite_copy);

It is also possible to use a section of an image as the source and another section as the target. The following call horizontally flips the left half of spritesheet into the right half:

img_hflip(img_sub(spritesheet, 0, 0, 16, 16), img_at(spritesheet, 16, 0));

This can also be performed directly on the VRAM on fx-CG 50 since the VRAM is an img_t in disguise.

Note however that performing a transform if the source and destination partially overlap (ie. they are not disjoint and they are not the same exact area either) is not supported for any transform and will always give an incorrect result!

Accessing image data and custom transforms

Every image's pixels can be read manually, and also written if the image is not read-only. img_t is a structure with a description of the image, and the following useful attributes:

  • img.width and img.height are the width and height of the image.
  • img.stride is the number of pixels on each row. Most of the time this is larger than img.width!
  • img.pixels is the pixel array.

The format is row-major order with stride. Here is how an 8x3 image with stride 12 looks like in memory. The numbers in the diagram correspond to indices in img.pixels.

 <------ img.width ------>
+-------------------------+-------------+
|  0  1  2  3  4  5  6  7 |  .  .  .  . |  ^
| 12 13 14 15 16 18 18 19 |  .  .  .  . |  img.height
| 24 25 26 27 28 29  etc  |  .  .  .  . |  v
+-------------------------+-------------+
 <------------ img.stride ------------->

For instance, the third pixel of the second row is img.pixels[14]. See how one needs to add img.stride to the index to go down one line, instead of img.width. Also, indices represented by dots such as pixels[8] are not part of the image and should never be accessed.

The reason why this format with stride is used is to support references to subimages. When creating a spritesheet of size 32x16, the stride is set to 32, as one could expect. But when we extract a 16x16 half, even though we are only considering half of each row, the rows are still 32 pixels apart from each other! This is why the stride of an image is often not equal to its width.

This also means that images are not contiguous in memory, so it is not possible to replace all the pixels in a single call to memcpy().

Here is how one would iterate over all the pixels of an image. The following piece of code turns all transparent pixels white:

img_pixel_t *px = img.pixels;

for(int y = 0; y < img.height; y++)
{
	for(int x = 0; x < img.width; x++)
	{
		/* Do something with px[x] */
		if(px[x] == IMG_ALPHA) px[x] = C_WHITE;
	}

	px += img.stride;
}

Pixels are of type img_pixel_t. The way to manipulate them depends on the platform:

  • On fx-9860G, img_pixel_t is uint8_t and its values are the colors from <gint/display.h>. The transparent color IMG_ALPHA is equal to C_NONE.
  • On fx-CG 50, img_pixel_t is uint16_t and its values are RGB565 colors. Opaque colors from <gint/display.h> can be used, except for IMG_ALPHA = 0x0001 which represents transparency. Note that IMG_ALPHA is not equal to C_NONE.

Issues and contributing

Bug reports and additions to the library (whether issues or PRs) are very welcome. Since this is a static and overall simple library, I don't mind adding whichever transform is useful to game developers.

Have fun playing around with this library!