Analytics Infrastructure

Greetings everyone,

Setting up analytics is our top priority inside of Finance right now. Not having the proper information at hand degrades the quality of decision making inside our DAO, among many other undesired effects.

To avoid wasting work and to come up with the best analytics infrastructure, it helps deciding what information we need to look at.

The first few that come to mind:

  • Financial:
    • general accounting (what was each transaction and why it happened?)
    • income statement
    • balance sheet (including treasury/NFT valuation, eventual debts we choose to incur)
    • cash flow statement (there aren’t deferred payments in crypto but we still need to keep track of things like funding rounds, all in all should be simple)
  • Operational:
    • compensation breakdown / evolution
    • latest trades of artworks from collections we own
    • details about our current collection (# of pieces by project, estimated valuation breakdown, liquidity etc…)
    • # of members / member activity by channel
    • KPIs of each area [TBD in governance]

The main question when designing analytics is where are the data sources and where is their destination?

For the sources, some of these are on-chain and some aren’t.

As for the destination, a data-warehouse was the first idea that ocurred to me, given it is a flexible solution which will allow us to evolve the schema and add private data if we want. This mandates some sort of ETL solution for on-chain data, which can be done with Etherscan API.

However, there’s this app called dune.xyz which turns the entire ethereum blockchain into a public queryable SQL database, bypassing the need for moving the data to a central database.

I believe that for financial reports we might be able to do everything on top of dune.xyz similar to makerDAO, as most of the data is on-chain (apart from transaction metadata).

However, to be able to do that, we must come up with a multi-vault scheme that will make general accounting possible.

That is, assigning specific vaults/wallet addresses to each area of the DAO (or per type of transaction, like compensation, project funding etc) which will allow for a simple SQL query merger to label and allow real-time tracking of the DAOs financials through dune.xyz, in a general level. It seems to me that is how markerDAO did their income statement, and sounds very logical in principle too.

This really should inform our pending vaults reform.

Click here for markeDAO’s amazing dashboard, demonstrating what can be done in dune.xyz.

Now click here for a simple demonstration of a Fingerprints collections breakdown I made in dune.xyz.

All that said, I’d like to hear your thoughts on the idea of not having a central database and just using Dune for analytics. Keep in mind all our metadata would be public, unless we go for the Pro package and keep queries private. It is a lot more expensive than setting up an off chain data-warehouse though ($390usd/mo/user).

Some operational off-chain data would remain forever separate from our on-chain stuff. I don’t see any major downsides to that for now, but perhaps there might be in the future.

Dune might be a useful temporary solution, though.

I’m not a data science expert so would really appreciate feedback from members who are. If you’re skilled in Dune (or just SQL, really), I’d also appreciate your help writing queries for the reports we need.

I agree that Dune is probably not the ideal solution for all of the uses cases outlined. Note that we wouldn’t need to hand craft an ETL solution for the blockchain data. Google BigQuery already runs an ETL pipeline on Ethereum daily and the data warehouse can be accessed through this dataset. I haven’t used it but it looks sufficient