In which I extricate myself from Flickr

I signed up with Flickr, the photo sharing platform often mentioned in a single breath with the term Web 2.0, in prehistoric times, in January 2005. This was a mere 11 months after it launched in February 2004.

The platform was positioned perfectly to field off newcomers Instagram (2010) and Hipstamatic (2009), but after the purchase by Yahoo! in March 2005 (for between a mere 22 and 25 million USD), Flickr slowly was driven into the ground, nearly succumbing to continuous mismanagement until it was thrown a lifeline in 2018.

Yahoo! passed the platform on to SmugMug, during the big selloff after Verizon had acquired Yahoo! itself. SmugMug, perhaps driven by a kind of nostalgia, stripping down some parts of the platform, and integrated management of the platform into their own organisation. SmugMug, like Flickr, focusses on photo sharing, but targets a more professional audience. Beyond a doubt, they can be credited with the continued, if precarious, survival of Flickr. Even today, it’s still among the 500 most popular websites in the world.

This continuing popularity, at least in part, is due to Flickr’s longtime support for Creative Commons licensing.

For developers, a huge draw of using Flickr was its powerful API, the programmatic way of ‘talking’ to the platform, allowing for mixing and mashing Flickr content with, and on, other platforms; An essential part of the ‘spirit’ of Web 2.0 was the drive for openness between platforms, mashing up different services and platforms to create something that was more than just the sum of its parts.

Lots has changed since those early days. Back then, now all-but-closed platforms like Twitter and Facebook were also providing access to large parts of their data through open APIs. 

Very early on, I started experimenting with the Flickr API. In 2006, I created a mashup where I used the contextual clues of individual images to geolocate them, something that, at the time, was not yet a feature of Flickr. I continued, creating a series of photo-based games, and, in 2014, LoremfFlickr.com, an image placeholder service for any platform, to show images on any topic, while respecting the licenses under which each individual image was released by their creators.

LoremFlickr became a huge success, eventually serving over 1 billion images. Only images released under a Creative Commons were included, with the author, and the license, embedded in the image, thus meeting the requirements of both the photographer, and Flickr. 

Then, in late 2024, LoremFlickr stopped working as expected. Seemingly, Flickr was now enforcing API usage limits. Then, in a difficult back and forth with Flickr support, being passed around to several staff members, and having to deal with a good dose of hemming and hawing from their end, I was told that Flickr was not to be used as an ‘image hosting’ service.
On the face of it, this might appear to be reasonable. After all, this can quickly result in large amounts of bandwidth being consumed by third parties. Except that the whole purpose of the API was exactly that, to show images somewhere that is not the Flickr website.

Soon after, I found that LoremFlickr was no longer seeing its API use limited, it was being completely blocked. More so, I discovered that all my projects that use the Flickr API were being blocked by Flickr. All of them only casually access the Flickr API, but, adding insult to injury, Flickr never took the time to notify me of any pending cancellation of my API use, so also never offering me the opportunity to meet their shifting requirements.

As the photos on my personal website, right here, were served through Flickr, this also meant that my photos were no longer accessible as part of my, now, 27 years of blogging history.

So, even though I have been a paying member of Flickr from nearly the very start, even with prices close to trebling over the years, it appeared that I now had no choice but to take my photos out of Flickr. But, as the obvious route for doing so is through the API, this was set to be complicated.

Sadly, with the platformization of the web, alternatives to Flickr, let alone open platforms, are very few and far between. Google can’t be trusted, Apple doesn’t offer an API.

Many years ago, disgruntled Flickr users left for a community-run competitor, Ipernity. Back in the day, I set up an account with Ipernity, but stuck with Flickr. Now, my Ipernity account no longer exists, and signing up with them is not possible.
The most suitable commercial alternative is SmugMug. But not only is that the platform which bought Flickr, meaning it was their staff that made the decision to block my API access, they also charge significantly more for usage.
One advantage is that SmugMug is really designed for pro- and semi-pro photographers to sell their work through their platform, allowing you, in principle, to recoup those costs. But, though I used to occasionally sell my photos, a shifting landscape, and AI, are changing this market profoundly.

Then there is the open source Piwigo, which provides an expensive hosted solution, and the option to self-host, and also the open source immich, for which some hosted solutions are available, like at PixelUnion. Another, good looking, solution is ente.

All these primarily are geared towards being replacements for Google or Apple photos, allowing you to not have to rely on Google’s or Apple’s cloud to store your photos. Ente does not offer an API, but immich and Piwigo do, meaning I could programmatically insert, and extract images on their platforms. However, sadly, as all these primarily function as photo storage solutions for individuals, they are not social platforms, meaning that the many Flickr-based solutions I built over the years, which rely on this social aspect, would not translate well to these platforms.

Yet, for being able to display my own photos on my own website, their APIs should suffice.

I decided to go with immich, and host with PixelUnion. I appreciate the advantages of fully self-hosting, but am also very aware that this is never as simple as advertised. And dealing with performance issues on your own server I find one of the least pleasant tasks out there. 
Piwigo also doesn’t have to be self-hosted, but the only hosted solution I could find was quite expensive. PixelUnion has the added little bonus that their servers are in the European Union, away from American jurisdiction.

It eventually took about two months to migrate my photos from Flickr to immich, and to have my website source my photos from my immich installation.
I managed the transfer by starting with a full export of all my data from Flickr, as I couldn’t use the API. This meant that I had to put together my own code for reading this export, uploading all photos, with their tags and albums, and then to write additional code to extract the right images with my blog posts on my site.

The whole process was very unreliable, and I ran into all sorts of problems.

• A few of the exported images were inconsistently named by Flickr, breaking my automated import.
• Immich can import Flickr (and other) exports, but, for Flickr, ignores the metadata which defines things like titles, tags, and albums.
• Front-end uploads to immich, through the browser, are not completely reliable.
• In immich, assets (images) do not have a title, but only a description.
• In immich, the description can not contain HTML.
• When I started with my import, I couldn’t find a way to query immich assets by tag. As my Flickr photos were connected to individual blog posts by tag, I had to convert these assets with the same connecting tags to albums.
• Immich doesn’t support collections. That is, groups of albums.
• Immich doesn’t support setting licenses for assets.
• Immich doesn’t allow you to rotate assets. This is problematic for older digital photos, or scanned images, which will not necessarily have the correct orientation after being uploaded.
• Replacing immich assets through the browser works counterintuitively, it never being clear whether the replacement has worked, and often it not working.

To run the import, I had to produce code that was able to access my PixelUnion server, and my Flickr export data. The Flickr export totalled something like 40GB, covering some 25000 photos, so uploading that to a server would be a bit tricky.

This is what I did:

1. Installed MAMP, to be able to easily run PHP on my own machine.
2. The Flickr export contains a small number of JSON files with all meta-data of all my photos. I put them, unzipped, inside a /json folder.
3. The Flickr export contains a bunch of zip files with only photos, about 500 per zip. For me, the smallest of these was just over 100MB. The largest over 4GB. I’d unzip around 5 at a time, putting all photos in an /assets folder.
4. I created a PHP script which read the folder of unzipped photos, then found the related JSON file, and then uploaded the image to immich, with the properties I wanted to include.
5. During this export, I converted ‘machine tags’ to albums. Machine tags are tags that more easily allow for grouping Flickr images when accessing them through the API.

This worked reasonably well, but was very slow. I also had to slow down the pace of the API requests, as otherwise the export often crashed the immich API. Even with the slow pacing, regularly the API crashed, which meant I had to adapt my code to automatically retry uploads repeatedly, to avoid the necessity of manual intervention. The process took weeks.

And then, after having uploaded around 15000 photos, I discovered that, for many of the uploads, albums, tags, locations, or titles were not correctly attributed to the assets.
So, I had to rewrite my code to also verify what were reported as successful uploads to immich, and then to store the assets which, after verification, and retrying, were still not tagged correctly.

Then, PixelUnion blacklisted my IP address. Though this was quickly resolved.

In the end, the whole process took about two months, and also involved making significant changes to my website. After all, my photos on immich are no longer accessible through the platform that hosts the images, but now can only be accessed through my own site, or not at all. So, my photography archive now allows you to view my photos by month, and by country, or territory.

Clearly, this is not a solution for everyone.

I still have some work cut out for me; a few of my projects rely on accessing my own photos through the Flickr API. It should be possible to update these to using immich.

But first, a break.

•  API  Flickr  immich  photography