How We Built Our DAO's Reputation System
How We Built Our DAO's Reputation System
Capturing contribution and streamlining project management
Reputation is an individuals’ most valuable asset. It’s their passport to opportunity, connections, and ultimately financial freedom.
For DAOs, this means reputations must be handled with great care. Reputation systems are a critical piece of community infrastructure. To fuel a DAO’s growth, a system which can identify who is trustworthy within the DAO, who is contributing value, and who is worth collaborating with is needed. The absence of one can make collaboration even harder than it naturally is.
This is doubly true for us at the SuperteamDAO. We’re a Service DAO building a talent liquidity pool for Solana founders. Our reputation system will form the basis of how projects are staffed, serviced, and ultimately evaluated.
In the rest of this essay, I’ll walk through how we think about reputation, current tooling options, and how we’ve built our own custom solution.
Reputation in Web3
Let’s start with the basics.
Reputation is inherently social. Your reputation exists only in the minds of other people.
Reputation is inherently relative. Your reputation is either stronger or weaker than other people. It doesn’t make sense to have a high reputation in the absence of anyone to measure your reputation against. Fortunately, since reputation is an infinite resource, it is not zero sum. Often, building your reputation alongside others creates a positive sum system for all involved.
Reputation is inherently dynamic. Your reputation grows and falls in accordance with your contribution and your output. Just because your reputation was high yesterday doesn't mean it will continue to be so the next day.
And up until the internet, your reputation was immobile. You could work at a company for years, but as soon as you join your next company you have to work to build your reputation from scratch. Sure, you’d have a resume, but that sheet of paper would always be a shallow representation of a complex reality.
Web2 made things better. Platforms like LinkedIn gave us the ability to visualize our networks, to garner recommendations, to showcase our work in a semi-public way. Instead of reputation being locked within an organization, it was now locked inside of a centralized platform.
In Web3, we are trending towards a more portable, more owned, and more accurate reputation system. There are a number of great teams working on this problem, including Rabbithole.gg and Questbook.app. These tools are awesome at providing a generalized on-chain resume that individuals can carry with them across the metaverse.
But within any particular DAO, things get a bit more complicated. It’s important to identify not just what someone is good at, but also how they fit into the context and opportunities of the DAO. First, DAOs are composed of strangers with varying degrees of contribution. A successful reputation system must capture who is actually getting work done. Second, since DAOs aim for decentralization, the reputation system should also be somewhat decentralized. And third, since DAOs are semi-open, their reputation systems must stand up in the face of bad actors.
For our specific use case, we wanted a system that would solve these problems and additionally fulfil three goals:
Identify high-performing talent so the DAO could efficiently match them to high paying project work.
Provide visibility to DAO members on which other members they should work with on projects.
Create a publicly visible dashboard that would enable DAO members to take their reputation with them throughout their web3 journey.
Current Tooling Options
To solve the reputation management issue, many DAOs rely on either SourceCred or Coordinape. There are a number of other really interesting platforms (e.g. Govrn), but because of their popularity, these were the two platforms we considered for SuperteamDAO.
SourceCred
SourceCred automatically assigns reputation points (called “Cred”) to DAO members based on particular actions. Right now, these include Discord reactions/messages, Discourse posts, and Github. You can then separately assign a monetary value to these points (called “Grain”), either with your own token or a stablecoin like USDC.
This is likely a great solution for some DAOs where tracking who is engaged is the primary focus. For example, who is actually committing code? Who is authoring governance proposals? Who is chopping it up in Discord? These are important and valuable questions for many DAOs.
However, for our Service DAO, engagement is not how we want to measure reputation. What we care about is quality of contribution. SourceCred, at this time, does not distinguish between different types of engagement very well. The questions our reputation system needed to answer are: Who is building and shipping work for clients? Who can be relied on to get work done? Who has what skills?
There may be overlap between these questions and Discord/Discourse/Github activity, but they are not 1:1. Since SourceCred couldn’t help for our specific use case, we had to look elsewhere.
Coordinape
If SourceCred maximises the automation of reputation attribution, Coordinape maximises the decentralization.
Coordinape gives every authorized member 100 reputation points (called “GIVE”) to allocate to other members of the DAO. Every member can elect to give away some, all, or none of the GIVE within a certain time period. This is the purest form of decentralized reputation management - every member is giving feedback to every other.
Like with SourceCred, you’re able to assign monetary value to GIVE as you wish. Typically this is done by assigning a pot of funds for a particular time period (called an “Epoch”) that are distributed based on the relative amounts of GIVE each person has accumulated. It’s entirely peer-to-peer, enabling a community to identify those who contribute on its own.
Philosophically, we loved the concept of Coordinape. Decentralized reputation management for a decentralized organization. Makes total sense!
However, there were three problems for our use case specifically:
1. Coordinape could reward the optics of working, rather than the work itself. Imagine I’m in a DAO with 99 other people. Of the 100, 10 are really contributing actively and the other 90 are at varying levels of contribution. I am personally super active in Discord, I’m commenting on every governance proposal, I’m on every community call. It might well seem to people that I’m the most active member in the DAO. But if the messages I’m sending are only memes, the only comments I have are inane and off-topic, and I just sit on the community calls in silence, my presence isn’t actually adding a lot of value. Of course, the 10 active contributors would know this and allocate me very little GIVE. But the other 90 people who aren’t actively engaged might end up allocating me a lot of GIVE, simply because I’m highly visible and my name is familiar to them. This can skew rewards to those who perform as contributors, without actually contributing. Ultimately this might result in poor cultural dynamics, where noisy performance is prioritised over quiet contribution.
2. Coordinape could be subject to bad actors. For example, if I join a 90 person DAO with 10 of my friends, we could all give 100% of our GIVE to each other and take 10% of the total allocated funding for the Epoch. This trick would only work in the short term, but it’s easy enough to pull off with even a small group of dedicated bad actors.
3. Coordinape is really good at recognizing who is generating value. But it is agnostic as to what that value is. It doesn’t distinguish or give visibility into who are the most impactful politicians in the DAO, who are the best writers, who are the best project leads, etc. Different people inevitably have different criteria for how to evaluate who is deserving of GIVE and who is not. As such, the GIVE totals represent what the community members think of each other generally, not specifically. For our DAO, it’s essential to know who is good at what skills so we can put them on projects where they can make an impact.
To be clear, I think Coordinape is awesome. None of these three problems are fatal, and many successful communities navigate around them. But for our specific needs at SuperteamDAO, Coordinape wasn’t the best option.
Since neither SourceCred nor Coordinape were a good fit for us, we collectively decided to build our own solution.
Our Custom Solution
First, an important philosophical caveat. Unlike some DAOs which aim for pure decentralization, the SuperteamDAO is aiming for federation.
In practice, this means that we want to have a large number of autonomous working groups without any hierarchy between working groups. No group is superior to any other, and no group reports to any other. Each group is able to decide what they want to work on, how they want to work, and when they want to work.
But within those groups, we expect to have clear leadership and centralized decision making. Every project team must elect a Project Lead who has the ultimate decision making ability. This Project Lead is the person ultimately accountable to the client and to the DAO for delivering the project.
Unlike a corporate environment, these Project Leads are a) chosen by their project teams and b) empowered only until the project ends. This ensures that hierarchies don’t ossify. The added benefit is that no project member is contractually obligated to work on the project if they don’t like the Project Lead. This creates a powerful incentive for Project Lead to be fair in his/her management of the project.
Using this federated organizational structure gives the SuperteamDAO the ability to assign reputation points at a project level. The Project Lead is the person closest to the ground and the person with the most at stake in the project’s success. Unlike a SourceCred based system, the Project Lead can assign reputation points for intangible and valuable work (as opposed to Discord/Discourse/Github actions). Unlike a Coordinape based system, the Project Lead can assign reputation points on a per-skill basis and ensure points go to those who actually are doing work, rather than play-acting.
With that in mind, here’s an overview of the entire process:
Step 1: Assigning Points to Tasks
On their own, those Project Leads may or may not have the skills necessary to determine how to allocate reputation points. After all, allocating reputation is a fraught activity that can feel intimidating, particularly for new Project Leads. We want to make it as easy as possible for new talent to lead projects, which means the DAO must supply some project management infrastructure around the Project Lead.
To remedy this problem, we’ve created a new function within the DAO called the Scope Squad. The Scope Squad is trained in, as you might guess, scoping projects. They are armed with templates (screenshots at the end of the essay) and best practices for how to break a project down. Together with the project lead, they embark on the scoping process.
The Scope Squad and the Project Lead are given a flat standard of 1000 XP (i.e. reputation points) per project to allocate to the different tasks. They are free to allocate in whatever way makes the most sense for them.
Some teams may just divide the XP up equally among team members. Others may divide up XP among the different skills involved in a project (e.g. 50% development, 50% design). Some teams may get more analytical and give more complicated/experience-intensive tasks more XP. The beauty of federation is that each group can experiment and decide on the best approach for them.
Each task is also assigned a “skill”, along with the XP. This enables the Project Lead to know exactly what kind of team s/he needs to deliver the project. The tasks, the associated XP, and the requisite skills are visualized on a notion board.
This board is helpful for the project team, but it’s even more helpful for new DAO members. Any new member can quickly see what projects need what work from people with their skills. They also know exactly how they’ll be recognized for working on that task. Given the importance of making contribution easy for onboarding people into the DAO, the benefit of this approach cannot be overstated.
Here’s an example to make this concrete. To build this reputation system, I worked with two brilliant DAO members, Abbas and Vijay. We identified two key deliverables: 1) a strategy document detailing the Reputation system and 2) a dynamic leaderboard showing who had the most XP segmented by type of skills. As a group, we decided that the development was 60% of the project and the strategic work was 40%. Vijay, the developer, got 60% of the total XP for the project (600XP), while Abbas and I each received 20% (200 XP each). Vijay could handle the majority of the Development tasks, and Abbas and I managed the strategy tasks.
Step 2: Multiply Points Based on Importance
This simple structure generated a new problem: not all projects are created equal. If every project starts with 1000 XP, then a 3 day project and a 3 week project offer the same pot of XP to divide.
To remedy this, we’re giving our Core Contributors team the ability to give a project a multiplier. Our DAO is still in its early days, and as such the Core Contributors are the only people who understand everything that is happening in the DAO and each project’s relative importance to the Season’s strategy.
This is not a long term solution. Over time, as the number of projects grows beyond even the understanding of the Core Contributors, we will need to find a new method of assigning the multiplier. My hunch is that we will organically develop guidelines for assigning this multiplier based on the project’s reach, impact, and clout. Once we have the guidelines, a community-elected team can be in charge of assigning the multiplier. But that remains to be seen.
Here’s how the multiplier works in practice: if a project is mission critical for the DAO, it can receive up to a 5x multiplier. For the Reputation System project, that might mean that Vijay is actually allotted 3000 XP, rather than 600. For another fun-but-not-essential project, the multiplier might only be 2x. And for a small project that is unimportant, that multiplier will only be 1x. The multiplier has the additional benefit of incentivizing people to work on the important projects (since they’re worth more XP), even if they aren’t as sexy or simple as other projects.
Step 3: Allocate Points and Outcome Based Bonuses
Once the multiplier has been applied, the project team gets to work. Crucially, their XP are only allocated once the project has been completed. This creates a strong incentive for team members to see the project through to the end and ensure something gets shipped. If they don’t ship, they don’t get XP. As Henry Ford said, “you can’t build a reputation on what you are going to do.”
The last bit of complexity is the addition of outcomes based bonuses. For example, our DAO is working on a project to onboard 5k new Phantom users. If that project exceeds its goal and gets 10k new Phantom users, all project team members will receive an additional 1.5x multiplier on their XP. This incentivizes people to focus on reaching outcomes, rather than just shipping any old thing.
A Look at Our Prototype
We decided early on to rely on Notion to power our Reputation System. While far from perfect, the DAO already kept all writing in Notion, so it seemed like a logical fit. Each project kicks off using the Scoping Meeting Template
Throughout the meeting, as tasks are added, we used the “Create Linked Views” option within Notion to show the same database in different ways, showing just the information relevant for that step in the meeting.
Once all the steps are completed, a board is generated with the tasks and the people they are assigned to:
We then invite a custom bot to each project’s Kanban board to fetch the data:
The bot then takes the Reputation amounts and updates the leaderboard dynamically. The leaderboard itself is sortable by skill. This allows DAO members and new Project Leads to quickly identify who the most consistent contributors in the DAO are.
Note: this is test data. The design is still a work in progress.
What the Future May Hold
This system is still a prototype. There are a bunch of ideas we’ve discussed but not yet implemented, including penalties for not shipping, time-based bonuses, XP floors/ceilings for team members, milestone based point allocation, and dynamic pricing of tasks.
But through the process, we’ve learned that a good reputation system ought to have a few key qualities:
Social (i.e. based on community opinions, not just the founders of the DAO)
Accurate (i.e. reflects the reality of contribution to a DAO)
Fair (i.e. not susceptible to bad actors, clear understanding of what contributes to reputation and what doesn’t)
Dynamic (i.e. regularly updated in a low-effort manner)
Portable (i.e. public facing)
The single most important lesson from our experience is to be specific about what kind of behavior you want to recognize. For a social DAO, it might be more about engagement and participation in events. For an investment DAO, it might be shepherding the best deals. And for a service DAO like ours, it’s about shipping projects that hit key outcomes. Once you know what you behavior is desired, you can easily design a system around it.
Ultimately, every DAO is different. Our needs are idiosyncratic and may not align with your DAO’s needs. If you want to talk about reputation systems in Web3 or have any specific questions, DM me @kashdhanda - I’d love to chat.
A huge, huge, huge thank you to Abbas and Vijay for developing these ideas with me through weeks of hard effort. Special thanks to Neil Shroff, Aditya Shetty, Devaiah Bopanna, and Sanah Dhanda for editing drafts of this essay and to Yash for designing the thumbnail.