Leo的恒河沙

I Made an Electronic Calendar in the Digital Age, Allowing Oil Paintings and Photos to Be Included - Minority#

#Omnivore

I made an electronic calendar in the digital age, allowing oil paintings and photos to be included.

Preface#

While sitting in a chair and daydreaming, I suddenly noticed that the desk calendar was still stuck on last month. In this digital age, physical calendars seem to lag behind our hurried pace; we rely more on our phones and computers to remind us of meetings, trips, and appointments.

The only calendar I cherish is the one I got from the owner on the day we officially got together, after we ran into a coffee shop to escape the rain. It had been less than 24 hours since we started dating.

Now, we have long since entered the halls of marriage. What gift should I give her on the eve of our anniversary? She happens to love tangible items, enjoys paper books, notebooks, and instant photos. How about a calendar? Of course, it has to be a bit different.

I hope this "book" of a calendar can last beyond 365 days, can turn pages automatically, can display to-do items, remind us of our anniversaries, and be aesthetically pleasing... So I created it, an ink screen calendar.

Calendar Features#

Functional Zones. The calendar is divided into three display areas: Image Area, Calendar Area, and To-Do Area. Every midnight, the calendar refreshes to update the calendar information. Whenever there are changes in the to-do items (addition, completion, deletion, modification), the calendar refreshes to display the latest to-do information and new images.

The image source for the image area can be set to randomly obtained online from the Metropolitan Museum, preset gallery, or user-uploaded images. The bottom left corner of the image area displays the title and author of the image. The calendar area shows the month, date, and day of the week. The to-do area displays the to-do items from Microsoft ToDo, sorted in descending order by "completion status" and "creation date," with completed items marked with a strikethrough.

Based on the aspect ratio of the image, the calendar automatically sets its orientation. The basic rule is that if the aspect ratio is less than or equal to 1, the calendar displays horizontally; if greater than 1, it displays vertically.

Interaction. In my previous article "Home Server Practice," I mentioned the multidimensional table Apitable, which is also used in this calendar. Interactions are implemented within the Apitable WebAPP, and the interactions available include:

• Display orientation settings: "Portrait," "Landscape," "Automatic";
• Calendar mode settings: Mode 1 "Image + Calendar + ToDo," Mode 2 "Image + Calendar," Mode 3 "Image";
• Image source settings: "Metmuseum," "Selected" (TOP1000), "Gallery" (photos);
• Upload custom images;
• Select and display specified images.

Settings interface and uploading custom images

Selecting and displaying specified images

Design and Production#

Overall Design Concept#

• Screen. I chose an ink screen because its display effect is the most natural and closest to paper.
• Data Update. The ink screen terminal is only responsible for receiving the final image data to be displayed; the acquisition and processing of basic data are completed on the server. This design is beneficial for maintenance (and remote sending of surprises) since the hardware will not be at hand during later use.
• To-Do Data. Must come from existing software, preferably with an API; I chose Microsoft ToDo.

Hardware#

The display uses a 5.65-inch color electronic ink screen module from Wisk, with 7 colors and a resolution of 600 × 448.

Display Calibration. The officially claimed seven colors are black, white, green, blue, red, yellow, and orange. Upon receiving the display, I found a significant color difference. Therefore, it was necessary to calibrate the actual colors of the display. Without a standard color card, I could only do a simple calibration: using a color printer to print the seven colors + neutral gray; taking pictures under uniform lighting, and using Lightroom to correct the photo colors with the neutral gray; then using the eyedropper tool to obtain the RGB values of the various colors on the ink screen from the photo.

The following are the values and display conditions after color calibration.

Varoom! -- Roy Lichtenstein from left to right: original work, image processed with dithering algorithm, actual photo of the ink screen

There are many options for the driver board: Raspberry Pi, Arduino, Jetson Nano, STM32, ESP32/8266. To simplify things, I chose the ESP32 driver board sold by the manufacturer, which has an onboard FFC socket.

Code#

esp32#

The code for the esp32 driver board is quite simple. It only needs to send an HTTP request to the server, and write the returned image data to the screen.

// StreamClient.ino
void setup() {
    wifiMulti.addAP(ssid, password);
    DEV_Delay_ms(1000);
}

void loop() {
    if((wifiMulti.run() == WL_CONNECTED)) {
      if(requestGET("newContent")){
        updateEink();
      }
    }
    delay(60000);
}
// Get image data
void updateEink(){
...
}
// Check for updated content
bool requestGET(String bodyName){
...
}

For computers, images are made up of pixels, and the size of the space each pixel occupies determines how many possible states (colors) that pixel can have. The simplest black-and-white image has each pixel occupying only one bit (1Bit), either 0 or 1, black or white. As colors increase, the space each pixel occupies grows larger, to eight bits, sixteen bits, twenty-four bits...

We have seven colors, so we need at least three bits of data to represent all colors, but to simplify calculations, we add a 0 in front, thus using four bits of data to represent the color of a pixel. This way, one byte (1Byte) can represent two pixels. Therefore, the number of bytes we write to the display = 600*448/2=134,400 Bytes.

For unknown reasons, despite the ample memory in the esp32, I could not create a full-frame image data cache and could only write in chunks: DEV_Module_Init();,EPD_5IN65F_Init();,EPD_5IN65F_Display_begin();``EPD_5IN65F_Display_sendData(gImage_5in65f_part1)

void UpdateEink(){
  HTTPClient http;
  http.begin("https://YOUR_SITE.COM");
  int httpCode = http.GET();
  if(httpCode > 0) {
      if(httpCode == HTTP_CODE_OK) {
          int len = http.getSize();
          // create buffer for read
          uint8_t buff[1280] = { 0 };
          // get tcp stream
          WiFiClient * stream = http.getStreamPtr();
          // read all data from server
          int numData = 0;
          String headString = "";
          while(http.connected() && (len > 0 || len == -1)) {
              // get available data size
              size_t size = stream->available();
              int c = 0;
              if(size) {
                  // read up to 1280 byte
                  c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
                  String responseString((char*)buff, c);
                  responseString = headString + responseString;
                  String temp = ""; 
                  for (int i = 0; i < responseString.length(); i++) {
                    char cAti = responseString.charAt(i);
                    if (cAti == ',') { 
                      if (numData < 67200){
                        gImage_5in65f_part1[numData] = temp.toInt();
                      } else if(numData == 67200){
                        DEV_Module_Init();
                        EPD_5IN65F_Init();                              
                        EPD_5IN65F_Display_begin();
                        EPD_5IN65F_Display_sendData(gImage_5in65f_part1);
                        gImage_5in65f_part1[numData-67200] = temp.toInt();
                      } else if(numData > 67200 && numData < 134399){
                        gImage_5in65f_part1[numData-67200] = temp.toInt();
                      } else if(numData == 134399){
                        gImage_5in65f_part1[numData-67200] = temp.toInt();
                        EPD_5IN65F_Display_sendData(gImage_5in65f_part1);
                        EPD_5IN65F_Display_end();
                        EPD_5IN65F_Sleep();
                      }
                      temp = ""; // Clear temporary string
                      numData++; // Increment array index by 1
                    } else {
                      temp += cAti; // Add character to temporary string
                    }
                  }
                  if (temp.length() > 0) { // Process the last number
                    headString = temp;
                  } else{
                    headString = "";
                  }
                  if(len > 0) {
                      len -= c;
                  }
                }
          }
      }
  }
  http.end();
}

Server Side#

The server is responsible for acquiring and processing art images, ToDo data, and calendar data, handling requests from the esp32, and managing interaction behaviors (apitable).

Art Image Acquisition

• Metmuseum. The Metropolitan Museum of Art is the largest art museum in the United States, with a collection of 3 million items, providing a selected dataset of over 470,000 artworks from its collection, which can now be used across any media without permission or payment. This can be accessed through their API. Here is a simple use case: parkchamchi/dailyArt. Through the API provided by Metmuseum, we can "randomly" obtain images from specified categories.
• Famous Oil Paintings. The images obtained online from Metmuseum may not necessarily be suitable for display on the ink screen in terms of color and size (too large or too small, colors too light). Therefore, I built a locally stored collection of world-famous paintings. I obtained the "TOP1000 Oil Paintings" from the most-famous-paintings website and stored them in Apitable. Below is the Python script.
• Holiday Images. Custom holiday and solar term-themed images stored in Apitable.
• Photos. Custom photos stored in Apitable.

import requests
from bs4 import BeautifulSoup
import csv

url = 'http://en.most-famous-paintings.com/MostFamousPaintings.nsf/ListOfTop1000MostPopularPainting?OpenForm'

r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')
artist=[]
images=[]
ratios=[]
for element_img in soup.find_all('div', attrs={'class': 'mosaicflow__item'}):
    artist.append((element_img.text).strip('\n'))
    imgRatio = int(element_img.img.get('width')) / int(element_img.img.get('height'))
    ratios.append(imgRatio)
    images.append(element_img.a.get('href'))

details=[]
rank = 1
for i in artist:
    painter = i[:i.index('\n')]
    painting = i[i.index('\n')+1:i.index('(')]
    ratio = ratios[rank-1]
    img = images[rank-1]
    details.append([rank,painter,painting.strip(),ratio,img])
    rank += 1

with open('famouspaintings.csv', 'w', newline='',encoding="UTF-8") as file:
    writer = csv.writer(file)
    writer.writerow(["Rank", "Name", "Painting","Ratio","Link"])
    for i in details:
        writer.writerow(i)

Image Processing#

Since the display screen only has 7 colors, images need to be processed to display in 7 colors. The Floyd-Steinberg dithering algorithm is very suitable for showcasing rich layers with a limited number of colors, allowing for better shadow rendering of the original image. It is particularly suitable for various use cases of electronic ink screens. It can also be easily implemented in Python.

from PIL import Image
def dithering(image, selfwidth=600,selfheight=448):
        # Create a palette with the 7 colors supported by the panel
        pal_image = Image.new("P", (1,1))
        pal_image.putpalette( (16,14,27,  169,164,155,  19,30,19,   21,15,50,  122,41,37,  156,127,56, 128,67,54) + (0,0,0)*249)
        
        # Convert the source image to the 7 colors, dithering if needed
        image_7color = image.convert("RGB").quantize(palette=pal_image)

        return image_7color

Based on the aspect ratio of the image, the calendar automatically sets its orientation, with specific rules determined by the image's aspect ratio (ratio). For images with excessively large or small ratios, the canvas is extended to adjust to an appropriate ratio:

• ratio < 0.67: Fill the sides with blank space to ratio=0.67, display horizontally;
• 0.67 <= ratio <= 1: Display horizontally;
• 1 < ratio < 1.49: Display vertically;
• 1.49 < ratio: Fill the top and bottom with blank space to ratio=1.49, display vertically.

Calendar Data Processing#

Calendar data mainly includes dates, days of the week, solar terms, and anniversaries. Solar term data can be obtained through 6tail/lunar-python. Anniversaries are manually set by me, and on the day of the anniversary, there will be a small firework. The color of the date numbers is derived from the current art image's tone:

def get_dominant_color(pil_img):
    img = pil_img.copy()
    img = img.convert("RGBA")
    img = img.resize((5, 5), resample=0)
    dominant_color = img.getpixel((2, 2))
    return dominant_color

ToDo Data Processing#

ToDo data comes from Microsoft ToDo. Since I also use ToDo data in other projects, managing it uniformly in n8n is particularly convenient. The retrieved ToDo data entries are sorted by status and lastModifiedDateTime and saved in the msgToDo.json file.

n8n retrieves ToDo data

Image Stitching#

Using Python's PIL library, I stitch together the art images, calendar, and to-do images, converting them into byte streams:

# concaten pic
img_concat = Image.new('RGB', (EINK_WIDTH, EINK_HEIGHT),WHITE_COLOR)
if DisplayMode == "Portrait":
    img_concat.paste(img_photo, (0, 0))
    img_concat.paste(img_date, (img_photo.width, 0))
    img_concat.paste(img_info, (img_photo.width, img_date.height))
    img_concat.paste(img_todo, (img_photo.width + img_info.width, img_date.height))
elif DisplayMode == "Landscape":
    img_concat.paste(img_date, (0, 0))
    img_concat.paste(img_todo, (0, img_date.height))
    img_concat.paste(img_info, (0, img_date.height + img_todo.height))
    img_concat.paste(img_photo,(img_date.width, 0))

buffs = buffImg(dithering(img_concat))
if len(buffs) == EINK_HEIGHT * EINK_WIDTH / 2:
    print("Success")

def buffImg(image):
    image_temp = image
    buf_7color = bytearray(image_temp.tobytes('raw'))
    # PIL does not support 4 bit color, so pack the 4 bits of color
    # into a single byte to transfer to the panel
    buf = [0x00] * int(image_temp.width * image_temp.height / 2)
    idx = 0
    for i in range(0, len(buf_7color), 2):
        buf[idx] = (buf_7color[i] << 4) + buf_7color[i+1]
        idx += 1
    return buf

Interaction#

As mentioned above, through the Apitable WebAPP, the interactions that can be completed include: setting display orientation, setting calendar mode, setting image source, uploading custom images, and selecting specified images for display.

• Settings completed through the WebAPP will take effect in the calendar during the next HTTP request;
• Uploaded images through custom forms will be added to the "Gallery" collection;
• Using the "mini-program" feature provided by Apitable, I wrote an image picker to select specified images for display, which will take effect in the calendar during the next HTTP request.

//YOUR_APITABLE_SPACE apitable space id
//YOUR_APITABLE_SHEET apitable sheet id
//YOUR_APITABLE_FILED apitable field id
//YOUR_WEBHOOK trigger process webhook
const datasheet = await space.getDatasheetAsync('YOUR_APITABLE_SPACE');
const record = await input.recordAsync('Please select a record:', datasheet);

const data = {
  datasheet: 'YOUR_APITABLE_SHEET',
  fieldid: 'YOUR_APITABLE_FILED' ,
  record: record.title
};

const response = await fetch('YOUR_WEBHOOK', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify(data)
});

Structure#

Based on the dimensions of the display screen and driver board, I designed a simple box-like shell, manufactured using 3D printing, with polycarbonate PC material, which has good toughness and heat resistance. The frame and back panel are connected with nuts, and injection-molded copper nuts are embedded at the connection points of the frame. The back panel has holes for USB cable access, support foot fixing, and hanging.

In Conclusion#

Thank you for enduring the tedious text in between and reaching this point. This is a rushed project with many rough aspects, and I hope to have time in the future to optimize and upgrade it. I also hope that my future self can still find the leisure to create some fun things.

Wishing you peace and joy every day.