Digital photo albums to survive a generation or more. Part 1

A photo from Władysława’s album with her father among members of the organization “Sokół”, in Silna, Poland, early 1920s.

A photo from Władysława’s album with her father among members of the organization “Sokół”, in Silna, Poland, early 1920s.

Introduction

My wife’s aunt Władysława always kept the family albums ready to be packed in a backpack, so that during WWII, when the family were expelled from their flat in Łódź and during further forced migration, the albums always traveled with her. This way the albums survived and we can now enjoy the family photos of several generations back. The durability of black and white photography and of the good quality, rag based non-acidic paper on which  they  were printed helped preserve  the hundred-year old prints.   

Color photos from 20 or 30 years ago did not fare as well. Organic dyes fade quickly, and some lost most of their colors. We are working on digitizing them, trying to digitally restore the colors, converting them to digital albums.

Creation of digital copies of the old albums has also another goal, in addition to preservation. The family was scattered around the world. Brothers and sisters who lived or worked in different partitioned regions of Poland before 1918 ended up in different countries; some returned to the reborn Poland, some settled in Germany, France, in the US, UK and elsewhere. A single album is not enough, but an online version can be viewed by many.

Which brings us to the question: how to create a durable, long lasting electronic album?

About 15 years ago, we installed Gallery2, an Open Source, Web-based photo gallery, which had everything that was necessary to build a collection of online albums. We used a desktop progm Picasa to organize and annotate the images. Now, Gallery2 and Picasa are no more (or not maintained, which is almost the same). This article is about our effort to rebuild the albums, so that they can survive at least for one generation (or, say, 25 years).

How can we predict the future, even just for 25 years, in the fast changing landscape of digital world? One factor that will help (and guide us) is a strong conservatism of programmers or builders of computerized systems. Let us take for example Unicode, an universal alphabet or mechanism to represent characters in almost all writing systems in use today. We will stress it later, because without Unicode one cannot really annotate pictures, for example photo report about a  travel from Łódź to Kraków to Košice to Hajdúböszörmény to Škofja Loka to Zürich to Besançon to Logroño. Unicode is about 25 years old, but was not the first, the Latin alphabet ASCII dominated computing for decades before. Today many programs and systems, even built quite recently, still do not support unicode. Similar conservatism affects formats for recording photographs (raster graphics). Some formats, like TIFF and JPEG, which were introduced 30 years ago, being first useful format became very popular. New standards like JP2, which are better in some respects, have a very difficult time to be accepted.

Goals of the project

It may be helpful to state the goals of the project. We start with a collection of photographs in digital form, which may be scanned photos or born-digital images. The images are organized into albums, perhaps matching original bound albums, or created from scratch. We also have metadata: descriptions, persons and places, dates, etc. associated with the individual images, and with the whole albums. Metadata may also include provenance, history and other events associated with them.

The goal of the project is to research and document the methodology to preserve the images,  their organization into albums, and all the associated metadata. We are looking forward to preserve it  for future generations rather than for a fleeting, one time viewing, which can be easily accomplished using social media today

Beyond the scope of this project is the preservation of bound albums and original photographs, scanning and storage of digital copies. Those topics have more or less extensive literature although some specific topics may be presented in this blog later.

Portability and Durability

The first feature important for this project is portability. What is portability? In simple words, a method to take your album, carry it with you and make available elsewhere, at different time, in different environment. Aunt Władysława’s album is an example of a portable object. It could be carried in a backpack, and was always available to read and enjoy - no special hardware or software necessary (with a possible exception of glasses).

Can we achieve such portability with digital datasets? It is easier if there is a commonly accepted standard of metadata storage. The data can then be exported using such standard, and then re-imported into another system (at different time, using different computer, different operating system, etc.). In genealogy the data can be packed into a GEDCOM1 structure, libraries have their  MARC2 standards, and archival finding aids can be stored in EAD3 format. In effect, we can save all painstakingly built album data, carry it in a pocket storage device and later re-load into a different system to be immediately usable.

There is no such standard for photo albums (as yet). The Gallery2 software that we used did have an import feature (from Picasa) but no export function. Popular and actively developed today web album systems like Piwigo or Silvermine can import data, but have no export capability. In order to achieve portability we will have to construct a system ourselves.

Portability is just one of several important features; paper albums are portable, but their durability in general is not so great. My uncles’ albums were all burnt in their Warsaw apartment, while they were away - one on forced labor in Germany, the other in a German concentration camp. We will talk about durability of digital albums later.

Lessons learned from the first edition of an electronic album

Fifteen years ago Picasa was a very popular desktop photo organizing software, and after it was bought by Google, it gained many improvements. It had many simple and more sophisticated editing tools, from cropping and straightening to color corrections, red-eye correction and many more. It kept the originals intact, one could build unlimited number of albums (even with overlapping content), export them in various sizes and formats. It had captioning4, tagging, geotagging and face recognition.

Gallery (later Gallery2 and Gallery3) appeared at that time as a perfect solution for building online albums. To begin with, it would accept an export from Picasa, including the sequence of pictures, captions and all. It could be easily installed in a standard website, and did not require a lot of maintenance. As it turned up later, upgrading was a complex process, but we did it very infrequently. The list of features was extensive, albums within albums, user comments, multi-lingual captions, user maintenance and many more. We have built two websites, one with current photos, and another with digitized old5 family albums, and published them among friends and family.

In a static environment such solution would work perfectly. However, the computer world in anything but static.The website software needs to be updated frequently, hackers constantly discover new “holes” that need to be plugged and designers, trying to keep up with the times, add features. In June 2014 the creators of Gallery put the project in “hibernation6”. In March 2016 Google announced, that it will not longer develop or distribute Picasa7. Local features of Picasa would still work, but the integration with Google services would gradually cease to exist. In other words, it was time to move on.

Before the closing the web albums in Gallery for good, we have downloaded and saved the images, as well as the database tables. The next step would be a hard look at the systems as it exists today, and ask what is likely to survive in the future.

Discussion

The first thought was to move the albums to a newer web-based system. We have reviewed the most popular systems, like mentioned Piwigo and Silvermine among others and installed Piwigo to test its features. Piwigo is a much more modern software with features similar to Gallery2. It has albums, user maintenance, can store dates without limitations, has a nice user interface. It can also import entire albums from several present-day desktop photo organizers, like commercial Lightroom or Open Source digiKam.

However, portability was still a problem. None of the systems had a mechanism to export images and metadata in a portable format. All of them are based on a database, each with its own construction. Restoring the data from one database and converting it to a different structure is a gargantuan task likely ending in a failure. There seems to be no path to create a portable album using any of those systems. And I would guess that those programs are not likely to survive and continue to be maintained for 25 years.

Another problem is the access to the photo albums or a question of privacy. The albums are not necessarily (or not always) for all to view - some may be aimed at a smaller audience. The user database and passwords seems like a good solution. However, many members forget their user names or passwords, and the older generations had difficulty with computers in general. Maintaining users and passwords is complicated, and many users abandon the website instead of trying to fix the problem.

In the end we decided to maintain the albums locally, on the desktop computer. Piwigo, Silvermine and other photo websites would still be a viable option for displaying the albums, but we we would not try to maintain them in those programs.

Below is the outline of the features of the proposed (and tested) solution; the details will be presented in the next part(s) of this article.

Portability

We will use the computer directory structure to organize the albums. Hierarchical file directories have been with computers for some 60 years or more8, and there are no indications that they will disappear any time soon. In addition, it is relatively simple to pack (for example zip) the whole directory into a single file, and transport it on a thumbdrive. We will put all files - images and additional data for a single album - in a single directory. To maintain sequence, we will use the directory sorting mechanism, and for example prepend the file name with sequential number.

Durability

Durability will be achieved by backing up the disk drive or its part. There are many backup products on the market, portable disk drives that maintain backups automatically and online backup solutions. As long as the data is copied (refreshed) regularly, and as long as copies are distributed (in the cloud or the hard drive passed on to family or friends) there are very good odds that they will survive for 25 years or longer.

Hierarchy

There was a temptation (and technical capabilities) to divide albums into smaller sub-albums aka sections or chapters, perhaps with different themes or dates. While it is not difficult to build, the portability of such structures is problematic. Databases maintain hierarchy in different internal structures, and there is no standard way to transfer the hierarchical data to the display systems. In the end we decided to simplify our albums, and flatten the structure drastically to one album - one directory (folder). If the album had some distinct parts, we would either divide it into several, or simply describe it in the album metadata.

Metadata

Metadata belong either to an individual photo or to the whole album and include: caption (description of the photo), date taken (if available), places and people in the photo. We decided against maintaining separate title and description for the image; as tempting as this was, it introduced another hierarchy, which is difficult to maintain. If there is a need to tell a longer story, we would keep it separately, and store it for future generations and facilitate the creation of different projects, such as books, etc.

The images can store metadata “inside9” - there are different schemas or standards for it, with better or worse support for unicode. Also, different photo formats use some of the standards. However, the most popular photo format (jpeg) and metadata standard (IPTC) work for the limited goals we have set, and we will use them to store photo metadata - with some caveats discussed in the next part of this blog.  

Album metadata does not comfortably fit in the photo. We decided to store it in a separate text file that would be stored together with the images in the same directory. Album title, description and any additional information like stories and comments would be stored there.

Album maintenance

As mentioned above, we have exclusively used Picasa to manage and edit the images. Its usefulness is diminishing to some extent - for a month now uploading the albums to Google Service (Picasa Albums or Photos) fails. One can upload the images manually, but the sequence is not maintained. There is a new Google uploader, which maintains sequence, and this would be a preferred path today for transferring the images online. Otherwise Picasa still works. There are alternative solutions, Adobe Lightroom for a yearly subscription, or Open Source digiKam, with many features (even though the interface is more complex). They help create albums and add metadata to images, the two basic requirements we have for this project. One has to remember the cardinal rule: maintain the images, albums and metadata locally; if any changes are to be made, keep them on your desktop.

Access

It is up to the album creator to decide if the album is public in the internet, or if the access is somehow restricted. With the existing technologies, there are several solutions:

  • Strict membership. The administrator approves a request for access, and only the user can view the albums. The onus of member maintenance is on the creator / administrator. We use this method in a family tree website.
  • Album sharing using the online service such as Google (also Flickr, iCloud, Dropbox, Amazon and many others). The user must sign-in to the service, and the administrator decides, how much sharing is allowed. For example in Google Photos or Google Plus one can allow or disallow re-sharing.
  • Album sharing by link.Google Photos allows one to share the album (or other resource) by providing a link with a long string of letters and numbers, such as this: Ll4kMjIKZFew7dDm1. It is ifficult to guess, but easy to share - no membership required. User is free to show the album to his family or friends.
    A similar feature can be added to a website, a marker asking the search engines (like Google, Bing, DuckDuckGo etc.) not to index the site. The site is still accessible and you can send its address to your friends and family, but Google will not index it nor display in search results.

Album Display

The main thrust of this blog is maintaining the photo-albums for the future, but we also needed to restore the albums now and test the solution. We decided not to apply photo album software like Piwigo, and use the online services, specifically Google Photos. The service is quite flexible, reads and displays some photo metadata and allows for album titles and captions. One can also edit images; but this should be avoided, as the round-trip is rather incomplete. Currently the free version of Google Photos reduces the size (resolution) of the photo. While perfectly useful for viewing now, it is not ideal for storage, which should always be of highest resolution. One can download the edited image, but it will be the size-reduced copy. Metadata added / edited online (e.g. caption, comments etc.) do not survive a round-trip. One can store high resolution originals in Google Photos, which counts against free storage limit, and may incur charges at some point, but the edited metadata will not be saved when downloaded. All this may of course change in years to come - or not, depending on the commercial interests of the online services.

The lesson is clear: feel free to use the online services for display but do not yield to temptation to maintain. Make modifications locally on your desktop.

We would also like to maintain a directory of published albums. Using the specialized photo-album software, the directory is automatically online. It is more difficult with online services. However, there are many ready-made website templates, and it is a relatively simple task to make a listing of the links to the actual albums; this is what we did in this iteration of online albums.

The solution we use for this iteration is hybrid: the individual albums presentation in Google Photos with access of the third type (see above), and for the directory a static website built on a template, with a marker not to be indexed by Google and other popular search engines.

In the next part we will discuss in more details the tools we use, and provide some how-to’s and methodology.

___________________

References

1) Description of GEDCOM in Wikipedia
2) MARC standards
3) Encoded Archival Description
4) The support for Unicode in Picasa was very good, except that while entering text using the most popular Polish “programmers” keyboard, after reaching the letter ą the whole text would irretrievably disappear. This quirk of Picasa was never fixed.
5) Unfortunately, we could not store dates of those images. The date system in Gallery was based on Unix, storing dates as a number of seconds that elapsed since January 1, 1970, and it did not accept earlier dates.
6) See the Gallery website
7) Picasa Retirement Announcement
8) See an article in Wikipedia on "Path"
9) See "The reverse side of a digital photo" in this blog.

Marek Zielinski, April 22, 2018

Explore more blog posts:

PARTNERZY
Ministerstwo Kultury
Biblioteka Narodowa
Naczelna Dyrekcja Archiwów Państwowych
Konsulat RP w NY
Fundacja na rzecz Dziedzictwa Narodowego
PSFCU
NYC Department of Cultural Affairs