Contents
Preface
Among the many functions of CF, there is a very practical function that I have only known in recent months. That is the protagonist of this article: R2 (object storage service provided by CF). Through R2, an important demand that individual webmasters have to face in the early stage of website construction can be met: image hosting.
Additional knowledge: What is an image hosting service and why do we need one?
An image hosting service is an online service specifically used to store and manage images. Users can upload images to the image hosting service, which will generate a link. Through this link, users can display the images on web pages, blogs, social media, forums and other platforms without directly storing the images on these platforms.
During the website construction process, image hosting can not only improve performance, but also reduce costs, simplify management, and enhance security, so it is crucial to the successful operation of the website.
In a broad sense, an image hosting service is a display effect: a platform that can display the stored image content ashttp(s)://*.png(or other formats of pictures) type of link. For example, when using WordPress to build a website, if you want to insert a picture in an article, you only need to upload the picture to the built-in media library, and a URL will be generated:
The URL format corresponding to the Monkey King image in the above picture is similar to
https://blog.tangwudi.com/wp-content/uploads/xxxx/xx/孙武空.jpgThis is the most basic and simplest website built-in image hosting.
However, this kind of website built-in image hosting has many disadvantages:
• Increase website server consumption
When the number of user visits is large, it will cause additional resource and bandwidth consumption on the website server (this can be alleviated by CDN).
• Increase the pressure on website server operation and maintenance
When there are more and more pictures, it will consume the disk space of the website server and is not conducive to operation and maintenance (batch picture uploading, site relocation, redundant site switching, etc.)
• Not conducive to the separation of website graphic and text content strategies
Often, different strategies need to be developed for website text content and images (caching strategies, hotlink prevention, hotlink prevention, etc.). This approach will make the configuration logic more complex and more difficult to choose when configuring the strategy.
Based on the above reasons, the image hosting mentioned in this article does not include this type of built-in image hosting, but only refers to image hosting that is independent of the website and provides services independently.
There are many ways to implement image hosting, including self-built servers (VPS, NAS, or open source image hosting systems such as chevereto), social media-based or free services, P2P distributed storage systems such as IPFS, object storage, etc.
Among them, due to its scalability and ease of use, the use of object storage as an image bed is very popular in many scenarios and is also the focus of this article.
Note: Building your own image storage also has its own advantages. For example, using chevereto to build your own image storage allows you to manage your own images and add watermarks to them. Interested friends can refer to the article:Docker series uses Docker to build your own image bed based on Chevereto,How to add watermarks to images using chevereto.
Additional knowledge: Object Storage (COS)
What is Object Storage?
Object StorageCloud Object Storage is a technology that stores data as objects. Each object includes the data itself, related metadata, and a unique identifier. Object storage is different from common file systems. File systems manage data in a hierarchical structure of directories and files, while the management unit of object storage is "buckets":bucket.
Object storage vs. file system
Compared with the file systems we are used to in our daily lives, object storage is somewhat abstract, especially in actual use, some object storage services provide similar functions to file systems, such as supporting multi-level "folder" structures. This may make the two look similar on the surface. However, they have great differences in their underlying working principles and application scenarios, which can be compared from several aspects:
1. Data storage structure:
• File System：Data is stored in the form of files in directories, which themselves can contain subdirectories, forming a tree structure. Each file has a path in the file system, such as /home/user/documents/report.pdf.
• Object Storage：Data is stored as objects, each of which includes the data itself, metadata (information describing the data), and a unique identifier. Object storage does not use traditional file paths or directory structures. Although some object storage services support organizing data in the form of "buckets" and "folders", these are conveniences provided at the logical level, and there is no actual file directory structure at the bottom.
2. User experience:
• File System: In the file system, users can intuitively create folders, organize files, and access files by path. The file system is very suitable for small files that require complex directory structures and frequent reading and writing.
• Object Storage: In object storage, users store data as "objects" and access them through unique identifiers (such as URLs). Although objects can be organized in "buckets" and "folders", these are only logical groupings and there is no actual directory structure at the bottom.
For example, in the object of a specific bucket in R2, there is an option to "View prefix as directory". If you do not check it, it will be the real appearance, and there is no such thing as a directory at all:
If checked, there will be a directory:
Therefore, directories and the like are nothing more than clouds and "eye-catching tricks" in the eyes of object storage.
3. Application scenarios:
• File System: Suitable for processing files that require complex hierarchical management and permission control, such as operating systems, office documents, code libraries, etc.
• Object Storage：Widely used in cloud computing scenarios to store large amounts of static data, such as media files, backups, logs, data lakes, web hosting, etc.
4. Access method:
• File System：Access data through file paths, which are intuitive hierarchical structures in the eyes of users.
• Object Storage：Access data through API, data is located through URL or identifier, the folder structure provided is only for the convenience of users to organize data, and actual access depends entirely on the identifier.
To summarize, although the "bucket" structure of object storage is similar to that of the file system in the user's perception, they work completely differently at the bottom layer: the file system relies on the directory structure, while object storage is based on the management of objects and metadata.
Although this distinction may not be clearly reflected in the user interface, it has a profound impact on performance, scalability, and usage scenarios: object storage is good at processing massive amounts of unstructured data and provides powerful data management and distribution capabilities, while file systems and block storage are more suitable for structured data processing that requires low-latency, high-performance storage.
Note: For some friends who like to tinker on their own and want to build their own object storage, this is also possible. You can refer to the articleDocker series builds a private COS platform based on minioHowever, this type of deployment is suitable for friends who have a public IP (now if you deploy in China, you also need a registered domain name). You can use a cloud host with a public IP or home broadband to build it yourself.
What is R2?
R2 is an object storage service provided by CF. Its purpose is to provide a cost-effective cloud storage solution. Compared with other traditional cloud storage services (such as Amazon S3), it has the following advantages:
- No export fees: A significant advantage of R2 is that there are no data egress fees, which means that users will not be charged when downloading data directly from R2, which is in stark contrast to other storage services (such as Amazon S3 and Tencent Cloud COS).
- S3 compatibility: R2 is fully compatible with Amazon S3's API, so users can easily migrate existing S3 data to R2 without modifying the code.
- Network integration with CF: R2 is tightly integrated with CF's global edge network, providing fast data access speeds and low latency, especially when combined with CF's other services such as Workers or CDN.
- low cost: R2 has relatively low storage costs, which is particularly attractive for large amounts of frequently accessed static content.
However, R2 is not a free service (hence why it is called "cost-effective" above), it just provides a free quota:
• storage: 10 GB of free storage.
• Class A Operation: 1 million free operations per month. Such operations include PUT, POST, COPY, DELETE, etc.
• Class B Operation: 10 million free operations per month. Such operations include GET, HEAD, OPTIONS, etc.
• Outbound traffic: 10 GB of free outbound traffic.
However, this free quota is completely sufficient for individual webmasters. For example, the data of my blog image hosting from August 1st to August 14th is as follows:
The total size of the images is only 430MB (it has been exactly one year since I started learning to build a personal blog at the end of August last year). At this rate, it will take more than 20 years to use up the 10GB free storage space. There are 2,000 Class A operations and 100,000 Class B operations, which are still a little bit away from the free quota limits of 1 million and 10 million respectively. As for the 10GB of outbound traffic, this is to cover a small number of special occasions. Under normal circumstances, the traffic routed through the CF network is free (besides, general content must first be supported by the free CDN), so most normal use will not generate outbound traffic.
Therefore, for individual webmasters, the statement that "R2 is free object storage" is reasonable, right?
Note: To activate R2, you need to bind an available international credit card (after all, it is not a "truly" free service, which is understandable), such as Visa, Mastercard, American Express, Discover, etc. This is the only threshold for using R2.
Why did CF name the object storage service R2? I did some research for gossip, and here are two possible reasons, the authenticity of which cannot be verified:
1. Pay tribute to pop culture
This name may refer to the robot character R2-D2 in Star Wars. R2-D2 is a very popular character known for its reliability and importance. CF may use this name to imply that their storage service is also reliable and plays an important role in the entire network. In addition, the name R2 is concise and easy to remember, which is also in line with CF's product naming style.
2. Benchmarking Amazon S3
CF named its object storage service "R2", probably to make some kind of connection or comparison with Amazon S3 (I suspect the inspiration came from F5 and A10):
• Continuation of alphabetic and numerical naming: Amazon S3 is a widely used object storage service, and the "S3" in the name stands for "Simple Storage Service." CF's choice of the name "R2" may be to continue a similar naming pattern while maintaining uniqueness and brand recognition.
• Competition and comparison：R2 is not only similar to S3 in name, but also functionally compatible with S3, and emphasizes that there is no data egress fee, which makes R2 a direct competitor of S3. Therefore, naming it "R2" may also be a clever way to imply that it is similar to S3 but provides differentiated advantages.
Build an image bed based on R2
R2 Image Hosting Tutorial (Beggar Version)
Follow the steps below to manually upload images to R2 and generate an https image link without relying on other tools.
The first step is to create a bucket:
The URL in the red box above can also be written directly as
https://demo.tangwudi.com/孙武空.jpg,becauseSun Wukong.jpgIt is uploaded directly to the "root directory" of the bucket (the quotation marks are because the so-called "directory" is just a smoke screen).
Then process other required pictures in turn, and finally convert the URL of each picture into the required format and insert it into the specified location in the article.
Note: If there is a "directory" in the bucket, the format of the link above will change accordingly. For example, if you put Sun Wukong.jpg in folder "1" and then upload the folder "1" directly, the link above will become:https://demo.tangwudi.com/1/孙武空.jpgThis is also a key point when migrating to an image hosting service. As long as the original image hosting service saves images and outputs URLs in the same way,/Directory 1/Directory 2/···/xxx.pngThis format can be directly migrated to R2 (of course, it can also be migrated to object storage services of other manufacturers).
This is the most basic way to use R2. It can be used, but few people use it this way (unless you occasionally upload a few pictures to get a URL), because you need to take a screenshot and save it locally, then upload the picture to R2, then generate a picture link on R2, and finally convert the picture link to the required format and insert it into the local article... Is it troublesome?
R2 Image Hosting Tutorial (Regular Version)
The conventional method of using R2 image hosting is to use a common editor (obsidian or Typora) + PicGo (image upload tool) with R2 to quickly implement conventional image upload and image address acquisition operations. Take a screenshot and paste the homepage of my blog in the article as an example:
Step 1, use screenshot software to take screenshots:
Step 2: Paste the image where it is needed in the article. A prompt box will appear:
This prompt appears because I have enabled the timestamp renaming function in Picgo. I won’t go into details here, just click OK in the red box.
Step 3: Then the previous screenshot appears at the specified location in the article and automatically generates an address in the corresponding format (here is the markdown format address), as shown below:
In other words, just take a screenshot, then paste it where the image is needed in the article written in a common editor, click confirm, and you can upload the image to R2 directly through Picgo's S3 plug-in, and generate a specific image address format at the corresponding location in the article according to Picgo's image format settings (Markdown, HTML, URL, UBB, custom). Compared with the previous way of using R2 image hosting, this method is much more efficient (that's why I call the previous method the beggar version), and this method requires the use of R2's API.
Note: For the specific settings of obsidian+Picgo, please refer to my previous article:Chevereto+PicGo+Obsidian is a series of tricks to achieve efficient image upload and image URL acquisitionHowever, the image hosting service I used in my article is chevereto, so the plug-in used by PicGo is the corresponding one for chevereto. If you change the image hosting service to R2, just change the PicGo plug-in to S3.
The process of creating an R2 API is as follows:
In my previous article, I used chevereto to build my own image hosting, so Picgo's plugin also uses chevereto's upload plugin, which needs to be changed to install the S3 plugin:
The S3 plugin settings are configured as shown below:
Using this method, screenshots are usually uploaded directly to the R2 image hosting service, which is also good. However, if there are some other requirements, such as adding watermarks to the images, this method will not work.
Note: There is a secondary development software "PICLIST" based on Picgo on github, which is said to support lossy/lossless compression and format conversion of images, as well as adding watermarks. However, I have not used it. If you are interested, you can try it.
R2 User Tutorial (Smooth Migration Edition)
Because I used chevereto to build my own image hosting and I am used to adding watermarks to my images, even though I use R2 as my main image hosting, I still want my watermarks on my images, but I don't want to mess around with it. In short, I don't want to change any of the configurations of obsidian, Picgo, and chevereto that I used to use, but I want to use R2 as my image hosting (I'm just so rogue), is that possible? Of course, you can. This is the core of this part, which is suitable for those who use chevereto to build their own image hosting and want to smoothly migrate their main image hosting from chevereto to R2.
The core of this smooth migration is "Rclone". Rclone is a powerful and flexible command line tool that is mainly used to manage and synchronize files and directories between different storage services. It supports a variety of cloud storage services, local storage devices, and other remote file systems, allowing users to easily copy, move, and synchronize files and directories between these different storage locations.
To complete the smooth migration from the chevereto image hosting service to the R2 image hosting service, we need to use Rclone to copy all the images in the local directory of the chevereto image hosting service to R2, and when there are updates to the chevereto image hosting service in the future, let Rclone quickly synchronize the updated content to R2 in an incremental manner. In fact, there are two image hosting services at this time, one is chevereto and the other is R2. Usually, images are uploaded directly to the chevereto image hosting service, and then synchronized to the R2 image hosting service using Rclone.
If you want to make R2 the main image hosting, you only need to give the original domain name of chevereto to R2 as a custom domain name. The detailed operation process of this part is not described in this article. Friends who are interested can refer to the article:Home Data Center Series Use rclone and cloudflare R2 to build Chevereto's remote disaster recovery image bed.
Precautions for using R2
It needs to be emphasized again:R2 is not a free service, but the free quota it provides is basically free for most personal webmasters under normal use scenarios., so in order to avoid the risk of use, please pay attention to the following points:
1. Free quota and excess fees
• CF R2 provides a certain amount of free usage (such as storage space and number of requests), but once these free quotas are exceeded, users will start to incur charges. Therefore, it is very important to monitor usage to avoid accidental overspending (normal usage is safe, the fear is abnormal usage).
2. Security and access control
• To prevent unauthorized access or theft, make sure your R2 bucket is configured with appropriate access control policies: You can use CF Access, IP restrictions, or custom permission policies to limit who can access your bucket and data.
• Avoid exposing API keys or other sensitive information in public repositories or code bases, as this may lead to unauthorized access.
In fact, there are other options for the public access part of R2, such as "R2.dev subdomain" or "CORS policy", etc., but the default options are fine in terms of security and suitable for most people, so I didn't mention them. Friends with special needs can change them by themselves (friends who don't understand should not tamper with them, for example, the R2.dev subdomain is closed by default. If this option is turned on and abused by others, the consequences will be a bit serious~~), as shown below:
3. Preventing abuse
• If your R2 bucket hosts publicly accessible content, consider setting bandwidth limits or using other CF protection mechanisms (such as Rate Limiting, WAF) to prevent malicious users from maliciously using your resources through a large number of requests (general personal webmasters don’t need to worry too much about this, after all, there is not much traffic anyway).
• Review your R2 usage logs regularly to quickly identify and respond to suspicious activity.
4. Monitoring and Alerting
• Keep up to date with your usage by setting up monitoring and alerting systems. For example, you can use CF's analytics tools or third-party tools to monitor storage and request usage, and set alert thresholds to receive reminders when you are close to the free quota.
5. Backup and Data Recovery
• Although R2 is a reliable storage service, it is still a good habit to regularly back up critical data to prevent data loss or unforeseen service interruptions (so, in fact, the combination of chevereto and R2 provided by the smooth migration solution can be regarded as a disguised backup)
Afterword
In fact, I was bothered by the so-called "smooth migration version" solution because I didn't know about R2 before, and I didn't dare to use Tencent Cloud's COS (I was afraid that the images would be hotlinked and waste CDN traffic), so the problem I considered before was how to implement the fault-tolerant mechanism of self-built Chevereto multi-node (see article:Tutorial on how to synchronize multiple nodes of Chevereto image bed in home data center series); The reason why we consider the fault tolerance mechanism of Chevereto multi-node is that we want to solve the problem of switching WordPress to Tencent Cloud lightweight server when there is a problem with the "home data center" (power outage or network outage at home) (see article:Home data center series uses cloudflare tunnel to realize automatic takeover of disaster recovery site when WordPress main site fails) (after all, the main image hosting service is also in the "home data center").
Later, I learned about R2. At first, I thought I could use it as a disaster recovery group bed in the "home data center" when it fails, but after using it for a long time, I became very satisfied with it and used it as the main image bed. If I had known about R2 earlier (the key is that I knew it was free), I would not have gone through so much trouble and took a long detour: I directly used R2 as the main image bed, regardless of whether I wanted to use multiple WordPress sites for fault redundancy (see article:Home data center series uses cloudflare tunnel to realize automatic takeover of disaster recovery site when WordPress main site fails), or use multiple WordPress sites for traffic load balancing (see article:Home data center series uses cloueflare tunnel to build multiple active redundant sites with traffic load balancing for dynamic blogs), you can use a consistent image hosting address, and you don’t have to worry about whether the image hosting address is available when a failure occurs.
Note 1: Backblaze B2's object storage service is actually similar to CF R2, with 10GB of free storage space. In addition, since Backblaze and CF belong to the "Bandwidth Alliance" (Bandwidth Alliance), so the outbound traffic between Backblaze B2 storage and CF's CDN service is free within the alliance, which means that if you use Backblaze B2 storage and distribute content through CF CDN, you don't have to pay for outbound traffic. The biggest advantage of Backblaze B2 is that you don't need to bind a credit card! So if you can't use CF R2 because of credit card issues, you can use Backblaze B2 as an alternative, and the effect is the same.
Note 2: In general, R2 image storage is used with CF's regular CDN (Cache Rules or page rules). The opening speed is not satisfactory, and generally it still takes 2-3 turns, which is acceptable. However, there is actually an optimization method, which is the worker optimization method in my CF series tutorial (VII). The code optimized for static content at the end of the article is used to accelerate the loading speed of images in the image storage. However, this method is subject to the limit of 100,000 worker requests per day in the Free plan. If there are many visitors, the quota may be consumed (after all, an article may have more than ten or twenty pictures, and the consumption speed of worker requests during optimization is more than ten or twenty times faster than that of an article). Therefore, sites with large traffic and many pictures should use this method with caution.
Note 3: In addition to being used as an image hosting service, R2 can also be used for other purposes, such as storing audio, video or any other format of files (the free storage space is only 10G anyway, so just use it up~), and then use it with CDN. However, although it is basically free, you should also pay attention to the degree. If the CDN traffic is too large and exceeds the normal range of personal use, it is not impossible to block the account (for example, a CDN traffic of several hundred TB~).
