It was only two weeks ago where I ended a blog post by saying that I wish I could move to Obsidian, but certain barriers were in place which kept me stuck using Notion. Man… how time flies, huh?

Yeah, I have a lot to explain.

To be clear, this was in no way brought on by that blog post. At the very most I intended to just find a way to use the Notion API to allow me to edit Notion items in Obsidian. Absolutely, under no circumstances, did I plan on exporting a 250k item database into Obsidian.

But that’s what happened anyway.

In order to understand why I switched, we have to once again dive into my radical weird productivity system… and how I found out the hard way that it doesn’t work with Notion.

How Stockpile Works (And Why Notion Is Incompatible)

Just so you know, this is now my third time explaining the Stockpile — there are much more detailed explanations on its dedicated blog post and video companion piece. I’m going to keep things as simple as possible here.

1. Put your life into 5-10 distinct buckets, based primarily on who you want to be and what you want to get better at. Mine are 3D Modeling, Art, Coding, Design, Game Development, Job Stuff (Things related to career or freelance), Language Learning (Japanese/Chinese), Music Production, and Reading. Job Stuff and Reading are more tangential buckets since almost everyone will have them.
2. Save everything to the stockpile. I used to separate this into a bunch of different categories like articles, books, audio files, images, etc… but I’ve realized now its a lot smarter just to say files, since books are PDFs and articles/notes are markdowns. That’s right, Stockpile is the first productivity system that relies entirely on file management.
3. Sort the stockpile. Every file in there has to relate to one of your buckets, or else it gets thrown out.
4. Every day, go down your list of buckets. Pick out 50-100 random items from each bucket. Your day is done when you have done this process for every bucket in the Stockpile. [1]

I love the Stockpile. It has become essential to my productivity and is the first time I’ve ever implemented a system that I think I’m going to keep permanently. After years of watching productivity gurus and reading their books, I finally stumbled upon something perfectly and uniquely me.

But there is one problem, and that problem is scale.

As you can guess, Stockpiles are big. Very big. Big to the point where you can get an enterprise-sized database just for a couple of things you want to do in the future. How? Well, let me give you a real example. In fact, it is this exact example which caused Notion to completely collapse on me:

[Image omitted]

In music production, there is something called “sample loops”. They are 15-30 second snippets of an instrument that you can essentially combine a bunch of together to create a brand new track (assuming you are not music-theory knowledgeable like me). The way these would work with a Stockpile is quite beautiful:

1. Save a bunch of sample loops
2. Listen to them during your daily MusicProd bucket, keeping the ones you like and discarding the ones you don’t. The ones you like you can put into a separate folder, called asset library.
3. With the weekly deadline system, where you have to make a new track a week [2], grab 25-50 random loops from the asset library. Use them to make your song. Bam!

So, I got to looking for loops. There are a few good sites for them — Looperman, FreeSound, and MusicRadar if you’re curious — but I came upon a rather interesting problem.

I ended up with 70,000 loops.

Now, for the sake of the process, this is good. It means I can make a virtually infinite set of music and almost never have to repeat the same loops (unless I do it by choice). And, if I’m going to be honest, I also don’t think this would have been bad for Notion if I had simply eased the app into this process (I was, at the time, only downloading about 2k-5k samples a week — only when they came up in the MusicProd stockpile). But the problem is that I didn’t have my “Stockpile as file management” eureka moment until after I had already obtained the entire loop library. So instead of importing 2k-5k files a week onto Notion (something that would be work, but not tear down the whole system) I just added all 70k files to Notion all at once.

You see, with any database, there is a theoretical maximum amount of files you can add to it. It is dependent on both file size (how long it will take to load each file) and file amount (how long it will take search for each file). If you’re building a query for 10 files, you’re going to get the query results almost instantaneously since it has to do very little searching and loading of each file. But as you get higher and higher up, the longer and longer it will take to do these queries until eventually it hits a point where the search just stalls out and gives an error. For enterprise-quality database searches like SQL this maximum is incredibly high, but for Notion… well, if you want to be precise, it’s probably somewhere between 200k to 250k.

Because that’s where it began to stall out for me.

So there I was, on a fresh Wednesday morning, realizing I was indeed quite screwed because my daily buckets were no longer loading. I needed a new solution, and fast.

Then I remembered my article on Obsidian.

Why Obsidian + Stockpile Is Only A Slight Disaster

When you look at it the right way, Obsidian isn’t a notetaking software — it’s a file management software.

All Obsidian is doing is taking a folder on your computer and allowing you to read/write edits to the files through its software. It does have a focus on Markdown, but all you need to get it to view all the other files in the folder is click a checkbox in the Settings. People use tools like Vim and emacs to create similar “file hubs”, but Obsidian is a bit easier for the average person to work with.

Because of this, it was a great choice for Stockpile 2.0. The issue with Notion (that many people are aware of, including me… but I tried to make it work anyway) is that its file management sucks. Uploading files is a complete pain in the ass: it has a cap of 200 files per upload, only successfully uploads content half the time, and has very little API support for them. To give Notion some credit, this is not a niche that they are trying to break into — but its something that’s necessary for us, and Obsidian (especially with its local directory support) does this way better. No more file caps, uploads are always successful, and anything we can do to automate a normal OS folder we can use to automate Obsidian. Score!

It is also worth noting that one of Stockpile’s most important features — randomization — is native within Obsidian. Building a PRNG for Stockpile in Notion is 1) a pain in the ass, and 2) part of the reason why the database collapsed. You see, the only way to get randomness in a Notion database is to utilize the item ID and the current time. Item ID is no big issue — but us utilizing the time formula means that our random generation is going to be changing every hour. This means that, every single hour in the day, Notion is going to have to calculate a new random ID for 250k+ items. This is silly and unneeded for the Stockpile, but there’s no other way around it. And, unfortunately, this is another niche Notion isn’t interested in.

There is, however, still one problem with Obsidian. And it happens to be the same issue that caused us to leave Notion:

How the hell are we going to sync the 250k files?

The Painful Exporting Process

The day that I attempted the export, I actually made a video detailing my progress (and pain) as it went on. You can watch it if you want to truly understand the amount of effort that had to be expended getting this whole thing working:

Since this is only the day of the export, most of the content in it is not how I’m currently running the system. Here’s more or less a step-by-step of what I went through:

1. Initially, I was in panic mode. I had a macro download and export each Notion item one-by-one (there is no bulk export for Notion) and push it into a Google Drive, which is where I intended to host my Obsidian Stockpile. I was able to get it working on the desktop and the Macbook, but the iPad was a no-go since it only supported iCloud drive sync. The phone solution was Foldersync Pro, which I initially had set up incorrectly and duplicated some files over 40+ times (!).
2. After finally getting the Foldersync working correctly, I realized there was a much smarter way to get (some) of my items out of the Notion database. While you cannot bulk export individual items, you can get a CSV of all their properties if you fragment the database enough (like ~10k items at a time). Some of these notes I knew were empty, just holding a URL in their properties. In the case of these notes, I could filter for has-URL, fragment out the database, then export it as a CSV and add it instantly to the Google Drive Obsidian Stockpile [3]. This worked! For the others, I kept the macro downloading them. It was also around this time I realized I was going to have to take the L on language-learning (100k+) and music-production (70k+) items and just start all over downloading them from their sources over time on the Obsidian Stockpile.
3. After the ~40k URL-only Notion items were ported to Obsidian, we began to have a problem. The Obsidian Stockpile was now too slow to use — much like with Notion, it had reached its theoretical maximum. There were other issues, too — as I had mentioned, the iPad was unusable, cutting off a great use case for the Obsidian Stockpile. The truth is, a DIY sync wasn’t going to cut it — we needed something built for Obsidian, and while I wasn’t itching to spend money on another cloud service, the cost/ROI on Obsidian Sync would still be much, much lower than the combination of tools I was using before. Fortunately, once I got everything ported onto Obsidian Sync, it worked — mostly.
4. At some point, I had another realization: while I could not export the remaining Notion database as a whole, I could soft export it — that is, grab the links to every item in the Notion database, and port those over as Markdown notes using the same system I used for converting the Notion URL CSV. This gave me the rest of the Notion links which I needed — we could now call the Notion database officially ported!

How It Stands Now/My Current Setup

Now, before I dive too deeply here, its worth noting that I’ve made vital sweeping changes to the Obsidian Stockpile as of today, so the chances of this becoming out-of-date within 24 hours let alone within a year are startling. Still, I think it will help others who want to implement Stockpile what I would deem as an essential plugin, as the needs of the Stockpile itself don’t really change that much. So here’s what I’m using

1. Binary File Manager – Okay, scratch that. I’m not using this plugin. This is the change made today that I mentioned earlier. I was under the impression that it was really hard to link attachments — particularly attachments not under Obsidian’s “embedded” cornucopia — without giving them their own dedicated Markdown file. So I used BFM, which essentially just made a MD file automatically for any file I added to the Stockpile. Unfortunately, there’s a big problem with this: BFM knows not to make MD files twice by holding the file name in a .txt file. If you lose that .txt file, then BFM makes a new MD file for every single file in the database. If you have 5k+ files like I do, this translates into an instant crash. In addition, if the .txt file is too big, Obsidian won’t load. In other words, BFM is unusable. Fortunately, it turns out my impression was false: files can be 1) put into a folder, and 2) shown in search for that folder. This means using raw files works just fine with Smart Random Note, which is all we need — the only real issue is that opening an unsupported file such as a .zip will open on the desktop instead of inside Obsidian, but this is a minor setback which we can handle.
2. Smart Random Note – this and one of its features (Alt-Q to grab link to a random note from search) pulls probably 95% of the weight for using Stockpile on Obsidian. It is a purely frictionless way to pick out those items from a bucket. Simply organize into folders, search the folder (say “path:reading/” or “path:design/”) and Alt-Q it up. Thanks Smart Random Note!
3. Text Generator – As I suggested above, the rest of these are not strictly necessary for the Stockpile — they’re just nice to have. Text Generator, for instance, gives you AI powers in your Obsidian — you can do API access to OpenAI/Gemini/Claude, or use a local model via LM Studio. It also comes with a text extractor tool that automatically grabs PDF text and youtube transcripts for you, and brings it right to the Obsidian page — this is a killer feature that could honestly be its own plugin, but comes built into Text Generator at no extra (plugin bloat) charge. Thanks Text Generator!
4. Spaced Repetition – Also a nice to have. Allows you to do Anki style review of your notes, or do full-on flashcards if you wish. I use note review for highlighted readings and flashcards for language learning. Keeps everything within Obsidian rather than having to refer to external tools — a huge deal for a Stockpile workflow.
5. Dataview – Allows you to SQL query your notes. I mean… you do know how many notes we’re dealing with here, so a bit of SQL magic goes a long way if you’re trying to find something in particular.
6. Tasks – Native to-do management in Obsidian. To be fair, Notion never had task management either — its funny because this is honestly a lot closer to Todoist functionality than any Notion template has gotten.
7. Charts – For the stats nerds, you can make charts! I use this to track the size of my overall Stockpile + buckets over time. There is zero functional use for this. It’s just fun.

Yup, that’s it. Just six plugins. To be fair, most other plugins will hard-crash Obsidian on boot with a directory of this size — but we don’t need anything else where we’re going!

[1] – The idea, of course, is that there is a to-do associated with every file. An image in the Art bucket should be used as a reference for a practice drawing. A note for an idea of a program in Coding should be executed on. If it cannot be turned into a to-do, it should also be kicked off the Stockpile.

[2] – It’s discussed in the video. Talking about deadlines in this post would get superfluous. I mean, you do know what a deadline is, right?

[3] – Using csv-to-md at https://github.com/astuka/obsidian_tools

[Image omitted]