VDP-109 [Funding]: Web3ification of TLDR longevity research peer review service

Proposals
1

One liner: we propose development of the web3 version of TLDR (the decentralized longevity review) to support:

1. content hosting on a production-grade database
2. web3 login for payment and identity management
3. upvotes, downvotes, and reputation accrual for authors and reviewers.

Overview

Previously we developed a web2 version of TLDR - (VDP-53) to enable paid peer reviews of longevity research. There were no costs associated with this as it was developed by Tim Peterson, who coded up the website as a side project paid for by his general role at VitaDAO. The web2 version was useful to demonstrate the functionality and as a marketing tool.

However, a web3 version is needed to realize the vision of the project. This project aims to enable researchers to have a better way to build their reputation. We believe this will be facilitated by web3 because of the superior incentive alignment mechanisms that web3 can offer. At a fundamental level, researchers’ reputation is currently tied to their email addresses. Take for example, one having a Harvard email address. If you don’t know the person you can almost immediately know the person performs credible research because of their email address (obvious caveats, but you get the idea). In the web3 world, one’s wallet address should signify that the researcher is important. Individuals’ reputation can be a collection of their activities at several organizations but importantly it is the individual to whom the value of their research output accrues. Not their organizations.

In terms of implementation, a key goal of this project is to initiate people into logging in with their wallet to replace email login. This will enable creators, in this case peer reviewers, to capture the value they create. Whereas in web2, the value is captured by the platform. One can’t accrue value to one’s email.

Problem

The Longevity Decentralized Review (TLDR) is an on-demand peer review service. The current prototype is live at https://longevity.review. It addresses 5 critical issues with academic journals:

  1. Researchers need to obtain peer reviews of their work in a timely and fair fashion.
  2. Researchers should get feedback on their research prior to entering the journal publication cycle.
  3. Reviewers should have the option of compensation for their efforts, especially if they make high quality contributions to the work (as viewed by the research community).
  4. The general public should be able to identify credible research without waiting the often yearslong (yes, plural years) review + publication cycle required by most journals.
  5. Journals should not profit off the free labor of researchers, nor paywall credible research content from the public.

It is important to note that TLDR does not interfere with the status quo. It is an opt-in mechanism for researchers to get quality feedback on their work and could function as a pre-journal submission screening mechanism.

Current Solutions

The research community has tried to remedy these issues mainly with (1) preprint servers and (2) social media promotion.

Preprint servers host unpublished research for consumption by researchers and the general public. These include BioRxiv, MedRxiv, and Arxiv. They solve the problem of distribution, though they fail to identify credible research and fail to provide researchers with feedback on their work. To reduce this problem, many researchers have made profiles on popular social media sites such as twitter and reddit, and use these profiles to promote their work or work they view as credible. The social mechanisms built into this site—namely likes/upvotes, retweets, comments, and follower/following lists—then provide the public with some basis for identifying credible research.

However, these communities are often unfocused and non-uniform. Thus, while social media promotion helps with content discovery, it very rarely provides researchers with high-quality reviews and does not address the problem of reviewer compensation whatsoever.

TLDR

How does TLDR currently work?

TLDR provides funding for researchers to review each other’s work.

Articles from websites known as “Rxiv” (pronounced “archive”) preprint servers, BioRxiv, MedRxiv, and Arxiv are auto-posted daily to the homepage. These “Rxiv” websites enable anyone to submit their research in the STEM (science, technology, engineering, medicine) disciplines for free. Once users register with email, they can upvote content and write a review by typing in a comment box. Users can search for content or browse the “all” page to find content to review. Authors can make a donation to TLDR to mark their post as “paid,” which incentivises reviewers to prioritize their work.

On TLDR, the incentive for peer reviewing these freely posted manuscripts is to receive a share of the donations given to TLDR. Additionally, VitaDAO is supplementing the reviewer payments with its $VITA tokens. The initial focus is on longevity related research, but in the future the scope could be expanded to other research areas.

As of now, TLDR is a product-research prototype that doesn’t support payments nor web3 login. It also doesn’t reorder content based on upvotes received. We propose implementing these features to upgrade TLDR into a real product.

What we propose to build

Summary

We propose to build the following infrastructure needed to upgrade TLDR to a fully-functioning product:

  • Web3 login for authors and reviewers.
  • A server which will store and host (1) posts in the form of links to preprint papers, (2) reviews on those posts, (3) upvotes/downvotes on both posts and reviews.
  • Payments in $VITA (and possibly USDC) by authors and to reviewers.

Description

Replacing the email login with an Ethereum wallet is a substantial novelty which aligns with VitaDAO’s mission. A connected address natively supports ERC20 transfers, allowing TLDR to compensate researchers with VITA tokens. This serves the dual purpose of (1) compensating reviewers and (2) distributing governance power to knowledgeable contributors with a record of providing high-quality feedback.

A full-fledged site would require a server to store and host reviews associated with papers added to the platform. These posts and reviews can also be uploaded to IPFS, but we believe a traditional database is better suited to task in its current state, since traditional databases offer flexibility and ease of development currently unmatched by IPFS. Of course, additional work can be done to backup any platform content on IPFS if VitaDAO deems this important.

Up and downvotes must also be stored and rendered, since this is what will eventually determine post ordering, will grow reviewer reputation and determine reviewer payouts. This iteration of TLDR distributes VITA to reviewers manually at the discretion of admins (precise criteria/algorithm to be determined in future, we will merely provide a simple tool to assist), so we feel that storing up and downvotes on-chain is unhelpful at this point in time. Doing so would require a smart contract—therefore an audit—so to keep costs down we have left this out of the proposal. As with posts, future work can be done to automatically upload up/downvotes on IPFS.

In the Additional Possibilities section, we outline additional work directions that have been left out of this proposal. This proposal is deliberately left simple in order to facilitate iterative development and reduce upfront costs to the VitaDAO treasury.

Timeline and Finances

We believe an MVP can be constructed within 2 months of starting work.

The sources of cost for this work are (1) compensation for contributors, (2) server costs, (3) professional advice on design and implementation. The bulk of development will be conducted by Petros Sideris, Jan Ole Ernst and Kyle Duffy, though we will be seeking the advice of Stefan Adolph and Tim Peterson for their respective engineering and product expertise. Over the 2 month timeline, we expect to pay at most $1000/mo for advice and contributions from Stefan, as well as at most $9000/mo for the remainder of expenses, which includes wages/rewards for 2 developers, and server costs.

We would like to share risk with VitaDAO by accepting a portion of compensation in VITA tokens. Accordingly, we propose a budget of 10,000 USDC and 5000 VITA per month, making the total maximal cost $20000 and 10000 VITA.

Additional Possibilities

The team would be available to develop additional features with additional funding if VitaDAO believes they are important. Specifically, we have considered the following additional features:

  • Mirror database (including posts and up/downvotes) to IPFS with a possibility of streamlining up/downvoting with snapshot.
  • Sybil resistance mechanisms, in the form of (1) staking or (2) linking web2 credentials or other unique identifiers to an Ethereum address (such as your ORCID ID) (details can be provided if requested in the comments, ZK proofs are also an option)
  • Reputation storage on-chain (requires audit)
  • Implement specific reviewer payout tooling
  • Automated payment dispersal to reviewers (based on a smart contract) (requires audit)

Team:

Petros Sideris. Experienced Software Engineer and CTO with background in Finance. Has co-founded a Web3 startup which delivered successfully two products.

Kyle Duffy (@kyleduffy) - Kyle did a Master’s degree in CS at Oxford and has spent the past year building in the web3 space. With Jan he competed at the ETHAmsterdam ETHGlobal hackathon and won with Retrox. He pivoted Retrox to DeSci and continued to develop it under a grant from the Ethereum foundation, releasing a product to bring ORCID credentials on-chain. Besides work in web3, Kyle has a history building production-grade software for hedge funds and trading firms. He spent time at Millennium Management ($58bn AUM) and Belvedere Trading where he built trade simulators and conducted quantitative research.

Jan Ole Ernst (@j-o-e) - Physics PhD student with a background in physics & philosophy and web3 enthusiast. Jan’s dedicated to improving research infrastructure and interned as a blockchain developer for SAP in 2019. He’s done a variety of hackathons and notably won the EthGlobal hackathon in 2022 with Kyle for building an on-chain retroactive public goods funding platform (Retrox) as well as some further DeSci infra. He also teamed up with Stefan at the DeSci London Hackathon in 2023 to build a privacy preserving web3 peer reviewing tool.

Tim Peterson (@timrpeterson) - Tim is a Longevity and Dealflow Steward at VitaDAO and CEO of BIOIO. He has over a decade of experience in both industrial and academic biological research. With over 11k citations, Tim has established himself as an expert on longevity research and is well-equipped to steer a product towards the needs of the longevity research community.

Stefan Adolf - Senior Web3 Developer at Molecule. Building IP capturing NFTs & their fractionalized counterparts. Dealing with metatx, cross chain messaging, DAO tooling and implementing great UX for our frontends. He has a fullstack developer background.

  • Agree
  • Please revise (comment below)
  • Disagree
0 voters
2

“Connect wallet” cannot replace logins soon enough.

I think trying out a smart contract to store some of the data would be good at this stage. I’m not sure TLDR deals with enough money/reputation at this stage for an audit to matter or be worthwhile ,let alone “required”…just releasing the code open source may be fine at this stage with a warning that it’s not audited. And the learning from trying to implement a smart contract will help keep future updates (when TLDR is bigger and has more potential users who would get mad at bumps in development) smoother.

NFTs can point to metadata that gets updated by the contract, which could be one way to store upvotes and downvotes on reviews. Minting the reviews as NFTs would also provide benefits like proof of authorship (ie the address that minted the NFT), and the data could be accessed by other apps (eg Orcid). It would also be closer to automating the payouts. Then it’s just a matter of sticking the metadata on Arweave or IPFS to move fully web3 native.

Not sure how gas would compare to the server costs. But this would be the time to build on Arbitrum, Polygon, or wherever we approved Vita to be besides mainnet.

3 Likes
3

Thanks for the feedback.

I get your point - an Ethereum address is not an identity. If you have an ENS a wallet connect is, I would argue, a fairly valid form of login though? On the other hand I do think there are also viable ideas to combine your address with any O-AUTH compatible web2 identity provider to provide the most seamless login experience and some form of Sybil resistance. ZK proofs of course are also a very promising idea here that could be looked at.

We definitely could implement a contract from the get go (ofc open source) and I totally agree that it will make future updates smoother. From the user side though I don’t think the experience would change radically, they would never see explicitly see the contract functions being called in the frontend anyway.

I do agree that if there aren’t many users an audit is a terrible investment. I would however say that its not great to use an unaudited contract when real money (or a token of real value) is being dealt with (no matter how little) and the potential loss of reputation is pretty big.

Also even if we deploy on cheap chains like Polygon/Arb… users don’t really want to pay gas (especially for something like up/downvoting), even if fees are low, so TLDR would need to cover user gas fees.

Gas costs would depend on the number of users and user interactions and what happens to fees on Arbitrum or Polygon in the long run, so is hard to say how expensive this might be.

We did think about using NFT’s and that is where the product would be heading longer term. How would you store metadata that is updated by a contract, if this is on-chain then this could be expensive and i probably not trivial to interact with, or what were you thinking of?

I was thinking of implementing proof of authorship by storing a signed message with the review so the reviewer can always verify authorship retrospectively. Snapshot for example, doesn’t currently store any votes on chain because it’s too expensive and impractical for the end-user, but the votes with the corresponding signatures are uploaded to IPFS.

I see the benefit of composability with NFT’s, but I think ORCID would probably be more comfortable hitting an API connected to a TLDR database which stores reviews, as opposed to fetching NFT reviews and linking them to authors. But of course I might be wrong.

I guess on the whole, we were worried that the more web3 infra we bring in from the get go, the more complicated the product would be more difficult&expensive to build and also we feared that it might scare away people with the right reviewing expertise but a lack of understanding of the blockchain world.

2 Likes
4

An Ethereum address is not an identity yet. One of the points of TLDR is to help make that more likely.

2 Likes
5

Agree with Tim that part of this is helping Eth addresses become an identity. Would rather avoid web2 identity providers where possible. Either ZK proofs if they work yet, or require some time of skin in the game for Sybil resistance.

Would hope the user interface gets smoother with web3 login. Personally, ‘connect wallet’ is the best log-in experience I’ve had in my life. I hate trying to remember passwords to everything, and don’t like signing into a password manager constantly.

Many ‘audited’ contracts get hacked, too. Reputation loss depends on the nature of the exploit and how sloppy the code was. But many people who care about audits still judge the devs based on how many and what kind of errors the audit reveals. So audits don’t mitigate reputational risk for devs much.

Agree gas costs would need to be mitigated. But if the contract stores some of the information, it might be cheaper than keeping that info in a database.

Like the NFT games. They store the data in a struct on the NFT and update the NFT with ‘storage’ function calls from the contract that allows minting the NFTs. Buildspace has the full step by step in their ‘Create a turn based NFT game’ build.

The hardest part may be only allowing someone to call a function for a given NFT once… which might be alleviated either by transaction history or by representing the user with an NFT as well, which would allow reputational storage and could track up/down votes too.

But this would need to be deprecated if there was a move to NFTs later on. Might be easiest to start with NFTs.

I’m not convinced anyone besides maybe Tim cares if TLDR is on their Orcid or not, and Orcid is incomplete in other ways. Would rather not let the dinosaurs hold us back. Plus, if the code is open source, this might provide the building blocks so Orcid can update to web3 and/or integrate with web3 with less effort.

Agree some of the web3 infra needs to be abstracted away for users and UX is important. But a short video ‘click these buttons in this order’ will fix some of that. to boomers, web2 and web3 are both confusing… just need to make it clear to them what buttons to click to make the magic happen.

1 Like
6

I agree ORCID is a dinosaur. It can be an optional part of one’s identity but we shouldn’t optimize for it.

4 Likes
7

I think the budget could be a bit larger, especially also on the VITA side to make everyone aligned.
Also better to have a bit of a margin of error for payments.
I’d suggest 10K $VITA and 20K USDC in total.

2 Likes
8

Is there emergent need for this ? Looking at the web2 TLDR page I see no public discussion or that any papers are being discussed.

There are also existing solutions to this, that are generic to multiple disciplines, so why would there be a need to have a niche specific one for longevity ?

1 Like
9

Ahh, I wasn’t even thinking of myself. I was thinking much more of the reputational loss to VitaDAO and the TLDR project. If things go wrong initially (hack, exploit…) it will be very hard to attract users to use TLDR in the medium-long term and wouldn’t look too good for VitaDAO on the whole.

Interesting, especially for storing up/downvotes (of which the there might me many) I’m not convinced that this is a particularly technically smooth solution where the NFT metadata would have to be updated every single time.

2 Likes
10

A hack would be embarrassing. But many protocols that get hacked were audited. Also not sure the protocol would be worth hacking if the TVL is low. Could keep most of the funds out of the contract, and only deploy prior to payday. Would limit the damage any one hack could cause. Then audit the newer version when it picks up steam and we add to it.

If it’s just a hack playing with up/down votes, it’s not worse than a Sybil attack.

2 Likes
11

Seems like a reasonable ask from a team which has delivered in the past, hence I am in favor of this proposal

4 Likes
12

Is the monthly vesting of 2500 VITA in effect? If not, it would be more appropriate to use USDC for payment. Utilizing liquid VITA for contributions doesn’t appear to be a reasonable approach.

1 Like
13

Yes vesting will be in effect. We’ll update the proposal to make that clear. Thanks for the feedback.

1 Like
14

gm. The best primitives to build this kind of tool must be evaluated while this implementation proceeds. Imo there are 2 major requirement implications that’ll affect the long term durability and acceptance. One being the “proof of excellence” of an account that relates to sybil resistance, privacy and (provably) linking external accounts. In a perfect world there would be an anonymous proof of excellence to prove that an undisclosed individual indeed bears a history of having reviewed, contributed or published to meaningful research achievements but that’s maybe not something you’d build in the first iteration. The other one being the question on where to store and how to incentivize review data to be evaluated by the original issuer or an evaluation comittee. Imo the most appealing tech stack for achieving this is ceramic / ComposeDB but still one has to prove and evaluate contributions so that the incentives can be paid out, ideally permissionlessly (aka “on chain”). I think it’s always a bad idea to store anything but monetary value (“tokens” or proofs of contribution / ownership) or timestamped commitments (eg hashes of external entities) on a blockchain. While EVM storage certainly has those capabilities and might at the time of writing be rather cheap on certain L2s, data availability should ideally be controlled with as few storage slots on any blockchain as possible. And non-coincidentally ceramic’s building blocks are a pretty good fit for this use case.

A word on “web3 as an identity” → an address (even when proven with a Siwe signature) should never be considered a replacement for an identity since wallets & addresses can change over time, while the person themselves stays the same. This is a vast topic, leading to year old discussions of SSI, DIDs and VCs and right now still many teams all over the world try to get that right. Some promising touchpoints right now might be Spruce ID, Fission or Ceramic Accounts but it’s quite hard to find an agreed upon, one size fits all solution here. If plain sybil resistance is mandatory, Gitcoin Passport or Privy could be options but that space also looks quite fragmented.

3 Likes
15

RSC doesn’t really have any proper web3 integration on their site, neither is there a wallet login.

I think we have a unique opportunity with TLDR because there is already a core community of people who are interested in longevity research and are also curious or at least open to using blockchain based software infrastructure so it is likely easier to get more traction on the platform in the first place.

The idea would be to expand the platform to other subject areas in the longer term.

16

This proposal has moved to phase 3 for live voting in Snapshot!
https://snapshot.org/#/vote.vitadao.eth/proposal/0x048a872bd518f2ffcf0441da9ab0069279277c6271c5b70e94c4a3d696b60dd4