2023 passed very quickly, as if the flow of time accelerated due to the emergence of ChatGPT. In fact, the biggest change for me this year was not communicating with AI, but how technology is fundamentally changing my view of imagery. In a way, I have embarked on a path of "experimental photography." The difference is that in the film era, experimental photography relied on darkroom techniques; in the digital era, it relied on post-processing tools like Photoshop; and now, I have begun to rely on various algorithms.
The first algorithms that prompted me to experiment and take this path were all related to 3D reconstruction. The first algorithm was NeRF (Neural Radiance Fields), and to implement NeRF, one needs to use SFM (Structure From Motion) and SIFT (Scale Invariant Feature Transform). Understanding these technologies and models took me over a week, followed by actual shooting and running the models. Looking back, the following results are what I obtained after about the first month.
The results were merely passable, but the shock of seeing the output was far greater than any equipment upgrade, even more than using a digital back for the first time. This is because it represents a leap from 2D to 3D, and because it was a month-long DIY process.
Then came continuous experimentation, refinement, process adjustments, algorithm improvements, and testing different parameter settings.
I believe that without the popularity of ChatGPT, the vast majority of my spare time in 2023 would have been dedicated to Photogrammetry and NeRF. However, the busy second, third, and fourth quarters of 2023 squeezed out all my shooting time, making me increasingly "theoretical": I could only quickly skim relevant papers or results without the time to optimize them through actual shooting.
From the perspective of photography itself, I am someone who values pre-production work far more than post-production: we have a lifetime to optimize our photos through post-processing, but perhaps only one chance to catch a fleeting light. Because of this, any new ideas require me to return to pre-production shooting for experimentation and optimization.
But just as post-production can remind one of points overlooked during the shoot, skimming papers and others' research can help optimize the shooting process and improve the quality of materials or data.
Thus, as the year drew to a close, a massive Christmas tree in a mall sparked my urge to shoot. I optimized the process and, without changing the algorithm, achieved results I believe surpass my previous work, serving as a period at the end of 2023.
Whether as an analyst, programmer, or photographer, I have a strong intuition telling me that 3D imaging is the greatest future opportunity. Digitizing our world and extending it infinitely within that space is a subject that, outside of exploring cosmic mysteries or human origins, is hard to rival. Regardless of my identity, I want to participate—and because I can only participate.
Technically speaking, 3D imaging is the foundation for autonomous driving and robotics. Fast and precise 3D imaging is the basis for all interaction and decision-making in the real world. 3D imaging is naturally the most important prerequisite for the next generation of film production and entertainment gaming. Furthermore, progress in industry, scientific research, energy, mining, construction, and many other research and engineering fields is highly dependent on this foundational technology, and that's not even including navigation.
Looking at actual progress, although many papers on 3D imaging were published in 2023, there hasn't been a breakthrough. NeRF was a 2021 achievement; the evolutionary directions in 2022 were accelerating modeling speed and improving image quality, but these were quantitative accumulations rather than qualitative leaps. The 3D Gaussian Splatting algorithm, as a derivative of NeRF, was a major highlight of 2023. However, both NeRF and Gaussian Splatting are essentially rendering algorithms that rely on 2D image alignment algorithms. Recently, some algorithms that do not rely on pre-alignment have emerged, but their use cases are still quite limited.
Objectively, 3D reconstruction models can be considered generative AI, or they might not be: many pixels in 3D space are "guessed" by the model, but they cannot be created out of thin air; they require a large number of 2D photos as a basis.
The higher the photo quality and the more precise the collection process, the higher the quality of the 3D generation. However, for higher quality, one must currently put in massive effort in the overall workflow—not just collection equipment and shooting processes, but also gallery management, image alignment models, and 3D modeling algorithms. These represent at least one camera, a PC with an NVIDIA professional-grade graphics card or better, two to three specialized software or algorithms, and a computation time that relates to image quantity and quality in an NlogN relationship.
So, when Apple released the Vision Pro, it's easy to imagine how shocking it was to see a feature that could achieve high-quality 3D imaging with just one all-in-one device. The term "spatial computing" even began to be widely used.
Although few have actually experienced the 3D imaging capabilities of Vision Pro, it deepens the understanding of products in the AI era: 1. All-in-One: simplifying complex processes around a core function to achieve "professional camera capability with point-and-shoot operation"; 2. Core models must be top-tier: second or third place might have some survival space, but beyond third, there is no commercial value because it will belong to the realm of open-source and free tools accessible to everyone; 3. Changing the interaction method.
To be honest, given my knowledge of Apple's hardware and software capabilities, I don't think Vision Pro's 3D imaging will be a qualitative leap over my current workflow. I often use iPhones with LiDAR combined with Polycam for 3D imaging, and the quality is indeed inferior to my current workflow. Vision Pro will certainly improve on this, but the improvement will likely be limited.
However, I hope to see similar hardware. If one day a company can launch a 3D imaging camera the size of a DSLR that completes the shooting, computation, and final imaging internally, I would be an early adopter without hesitation. I'm even planning to DIY such a device myself.
Fortunately, in Q4 of 2023, such a device appeared. Unfortunately, I might have missed another entrepreneurial opportunity to create a product.
This is the Miraco by Ravopoint. It's a 3D scanner—an all-in-one device that handles the entire process from collection and computation to 3D imaging without needing any other equipment.
I participated in the crowdfunding immediately. After months of waiting, I received the machine on the same day I returned to Singapore.
I won't do an unboxing video; I just took a few photos for a visual comparison.
The operation is very simple. Although I'm still getting used to the device, the near-automatic operation, fast computation, and high precision have already amazed me. To fully demonstrate its capabilities, I used the device's screen recording function; the videos below show the actual operation process at original speed.
The entire process used no equipment other than the Miraco. The next video shows several small models I built entirely through the machine itself, also a screen recording of the actual process at original speed.
I am still in the break-in period with the machine, so this isn't a strict review, but I can briefly summarize my experience from the weekend:
- In the AI era, these hardware-software integrated All-in-One devices are a key trend, perhaps the most important one; once the barrier is established, it's much higher than pure software applications.
- The "ChatGPT moment" for 3D imaging is arriving quickly as hardware and models advance together.
- The product's maturity is quite high; it is indeed the most accurate and convenient complete product currently available, though it still has flaws.
- There are some firmware issues, particularly with the PC connection.
- Indoors and at close range is where the machine excels, but performance in outdoor environments is poor, especially in strong light where dynamic range issues make it almost unusable. I believe this is due to both camera hardware limitations and immature firmware, which should improve with updates.
- This product is just the beginning. If Ravopoint, as a leading 3D scanner company, provides a better developer ecosystem—such as opening firmware model interfaces to allow users to load the latest Gaussian Splatting models—the capabilities and application range of such products will expand tremendously. After all, most users likely have some programming skills and are familiar with 3D imaging algorithms.
Admittedly, as a first-generation product, it cannot meet all my 3D imaging needs, but it is at least a huge breakthrough: 1. Paired with a 3D printer, it can easily complete the workflow from collection, computation, modeling, and editing to output; 2. If humanoid robots and autonomous cars are large edge computing devices, Miraco might be the first complete portable small-scale edge (generative) AI computing device. Its sales volume won't be massive, but it can provide sufficient hints and leadership for startups looking for AI application opportunities; 3. In the next five to ten years, 3D imaging will be a market at least on the scale of professional digital cameras. This industrial chain is long enough, the imaginative space is large enough, and the application scenarios are numerous enough—and now, everyone has a chance.