The moral embeddedness of cryptomarkets: text mining feedback on economic exchanges on the dark web

Abstract

Reputation systems promote cooperation in large-scale online markets for illegal goods. These so-called cryptomarkets operate on the Dark Web, where legal, social, and moral trust-building mechanisms are difficult to establish. However, for the reputation mechanism to be effective in promoting cooperation, traders have to leave feedback after completed transactions in the form of ratings and short texts. Here we investigate the motivational landscape of the reputation systems of three large cryptomarkets. We employ manual and automatic text mining methods to code 2 million feedback texts for a range of motives for leaving feedback. We find that next to self-regarding motives and reciprocity, moral norms (i.e. unconditional considerations for others’ outcomes) drive traders’ voluntary supply of information to reputation systems. Our results show how psychological mechanisms interact with organizational features of markets to provide a collective good that promotes mutually beneficial economic exchange.

1. Introduction

For mutually beneficial market exchange to take place, traders need to overcome the problem of cooperation arising from the information asymmetries between them (Beckert 2009). It has been argued that legal, organizational, social, and psychological mechanisms promote trust formation and cooperation in markets (Milgrom, North, and Weingast 1990; Diekmann and Przepiorka 2019). However, online markets that connect anonymous traders across the globe largely evade legal assurances and the embeddedness of these traders in ongoing personal and business relations (Przepiorka and Aksoy 2021). Such markets, arguably, rely more on organizational facilitators of cooperation and traders’ moral norms (Bolton, Greiner, and Ockenfels 2013; Przepiorka 2014; Ahrne, Aspers, and Brunsson 2015; Kuwabara 2015).

Moral norms have been regarded as the foundations of trust (Uslaner 2002) and trustworthiness (Riegelsberger, Sasse, and McCarthy 2005; Gambetta and Przepiorka 2014) facilitating mutually beneficial market exchange. Moral norms embody individuals’ other-regarding preferences that are not contingent on others’ expectations and sanctions (Bicchieri 2006; Greene 2013; Horne and Mollborn 2020). Before anonymous online markets existed, it has been argued that a generalized morality would be needed for cooperative market exchange to take place in the absence of legal and relational norm enforcement mechanisms (Granovetter 1985; Platteau 1994). However, the possibility to “engineer” trust by means of today’s electronic reputation systems, which collect, aggregate, and transmit information about traders’ past behaviors (Resnick et al., 2000; Bolton, Greiner, and Ockenfels 2013; Diekmann et al., 2014; Ahrne, Aspers, and Brunsson 2015) suggests that moral norms are not essential in promoting cooperative exchanges in online markets (Przepiorka 2014). In online markets that employ reputation systems, traders’ trust and trustworthiness primarily hinge on these traders’ expectations of gaining or losing reputation, rather than their regard for the outcomes of others.

Here we argue, instead, that the essential role of moral norms in supporting cooperation in anonymous online markets has not disappeared, but shifted from helping solve the trust problem at the stage of market exchange to overcoming the collective goods problem at the stage of leaving feedback on individual experiences with completed exchanges. The effectiveness of reputation systems in solving trust problems between sellers and buyers depends on traders’ sharing of feedback about their experiences with other traders, products, and services. However, feedback information is costly to produce and becomes freely available to everyone in the market. This confers reputation systems the status of collective goods that are susceptible to free-riding (Bolton, Katok, and Ockenfels 2004; Diekmann et al., 2014; Diekmann and Przepiorka 2019). Yet, studies show that traders in online markets leave feedback in high numbers (Diekmann et al., 2014). Identifying the drivers of traders’ feedback is thus critical to our understanding of cooperative market exchange. Starting from the conjecture by Granovetter (1985) and Platteau (1994), we ask the following question: To what extent do moral norms (next to other motives) drive traders’ feedback after completed online market transactions?

We address this question by means of automatic analyses of 2 million feedback texts accompanying transactions in three large illegal online marketplaces—so-called cryptomarkets. Cryptomarkets are situated on the Dark Web, a portion of the internet accessible only with the help of anonymizing software (Gehl 2018). By addressing our research question in an extra-legal context, we make several important contributions to sociological scholarship concerned with collective action, the organization of markets, and quantitative research methods.

First, we investigate the interplay of organizational and psychological mechanisms in promoting cooperative market exchanges largely unconfounded by legal and social mechanisms (Beckert and Wehinger 2013; Simpson and Willer 2015). This, as such, already demonstrates how the increasing popularity of online markets has shifted the role of psychological mechanisms in promoting cooperation in markets from the exchange stage to the feedback stage. However, cryptomarkets present a highly uncertain environment where trust needs to be established in the absence of legal assurances, verifiable identities, or codes of conduct present in conventional illegal markets (Przepiorka, Norbutas, and Corten 2017). What is more, the necessary drivers of information sharing in online communities identified by Kollock (1999) are absent. The reputation systems of the cryptomarkets that we study provide very little opportunity to engage in ongoing interactions with other users leaving feedback, attribute feedback to the users who provided it, or have contributions to the feedback system rewarded by others. By focusing on cryptomarkets, we thus unveil a major conundrum: How is it that cryptomarket traders still cooperate and contribute to the collective good of a reputation system by leaving feedback?

Second, and to address this question, we provide the first insight into the motivational landscape of reputation systems of three large cryptomarkets: Silk Road, AlphaBay, and Hansa. We devise a comprehensive theoretical framework by synthesizing insights from the behavioral and experimental social sciences in general and from previous research on traders’ motives to leave feedback in online markets in particular. Our theoretical framework constitutes the starting point for our analysis of feedback texts and allows us to contextualize the role of moral norms within the set of motives traders may have for sharing transaction-specific information for the collective good of reputation systems.

Third, to infer traders’ motives for leaving feedback, we combine manual and automatic text analysis of a large number of feedback texts. Our approach allows us to capture more motives compared to previous studies, explore the co-occurrences of different motivations in individual feedback messages, and, most importantly, tie motives for feedback to actual transactions in online markets. Our approach, thus, circumvents the limitations of existing survey studies which explored hypothetical situations and relied on convenience participant samples (Hennig-Thurau et al., 2004), as well as the challenges of inferring motives through interviews (Small and Cook 2021). Other studies have used digital trace data on the timing of quantitative feedback to assess market-level feedback patterns (Dellarocas, Fan, and Wood 2004; Diekmann et al., 2014). Yet, inferences based on aggregate behaviors cannot disentangle different motives that may be at work at the same time and to different extents across individuals. Our approach enables us to investigate the relative importance of motives and motive co-occurrences for promoting cooperative market exchange.

Finally, we demonstrate how our approach makes an even deeper understanding of the role of moral norms in markets possible. We use the fine-grained information on traders’ motives for leaving feedback to shed new light on two challenging situations that might prompt morally motivated action (Stets and Carter 2012) in markets: the entry of new sellers and the exclusion of untrustworthy sellers.

2. Theory

2.1 Assurances of cooperative market exchange

In markets, sellers often have private information about their products and services that buyers cannot access before completing the transaction. Such information asymmetry creates a trust problem if exchanges are sequential and buyers have to pay sellers before these sellers provide their products or services (Akerlof 1970; Coleman 1990; Kollock 1994; Reuter and Caulkins 2004). However, several assurance mechanisms reduce buyers’ uncertainty about seller trustworthiness¹ (Cook, Levi, and Hardin 2005; Diekmann and Przepiorka 2019; Ladegaard 2021): first, a functioning legal system makes sellers’ cooperation formally enforceable (Platteau 1994; Nee 2005). Second, buyers’ and sellers’ embeddedness in a network of ongoing personal and business relations makes sellers’ cooperation informally enforceable (Granovetter 1985; Buskens and Raub 2013). Third, sellers’ cooperation is enforced via formal rules set up by the market platform providers (Milgrom, North, and Weingast 1990; Ahrne, Aspers, and Brunsson 2015). Finally, certain values, preferences, and beliefs held by sellers make these sellers’ cooperation intrinsically enforceable (Platteau 1994; Kuwabara 2015).

The formal enforcement of cooperation through legal entities can be ruled out in both conventional and online illegal markets (Jacques and Wright 2011; Beckert and Wehinger 2013 although see Gambetta and Przepiorka 2019).² Under conditions of high uncertainty, traders in conventional legal and illegal markets alike rely on personal and kinship ties to informally enforce cooperation (Kollock 1994; DiMaggio and Louch 1998; Gambetta 2009; Jacques and Wright 2011; Beckert and Wehinger 2013; Moeller and Sandberg 2015, 2017; Ladegaard 2020). However, in online illegal markets traders engage with each other behind a veil of pseudonymity (by relying on usernames) while their real identities are kept anonymous. Even though usernames allow traders to occasionally rely on transaction networks built through repeated interactions (Axelrod 1984; Buskens and Raub 2013), these networks cease to be relevant in promoting cooperation as cryptomarkets grow in size and complexity (Duxbury and Haynie 2021, 2023). The limitations of legal and relational assurances in cryptomarkets then put a greater burden of promoting cooperation on market-level organizational assurances and traders’ moral norms.

The organizational assurances of cooperative behavior are facilitated by the technological infrastructures of cryptomarkets (Morselli et al., 2017; Diekmann and Przepiorka 2019). Reputation systems are an organizational assurance that takes a central role in supporting the functioning of cryptomarkets by providing traders with a means to establish each other’s trustworthiness (Pavlou and Dimoka 2006; Cook et al., 2009; Przepiorka and Berger 2017).³ The existence of reputation systems allowed cryptomarkets to move beyond reliance on trust devices available in conventional illegal markets, including monitoring and gossip rooted in small interpersonal networks and character-based reputation engendered in individuals’ personal characteristics or their membership in groups and organizations (Gambetta 2009; Beckert and Wehinger 2013; Diekmann and Przepiorka 2019). As a result, the costs associated with violence, limited choice of suppliers, and lack of information on the quality of traded goods present in conventional illegal markets are also reduced in cryptomarkets (Reuter and Caulkins 2004; Bakken, Moeller, and Sandberg 2018).

How do reputation systems help overcome information asymmetries between cryptomarket traders? Reputation systems establish traders’ reputations for trustworthiness based on the feedback these traders’ interaction partners leave after completed transactions. To ensure reputation is costly to obtain—and thus risky to lose—most online markets only allow feedback from traders who have completed a transaction. To be chosen as trading partners and build their reputation, new sellers must accept lower prices when entering the market (Shapiro 1983; Friedman and Resnick 2001). Once their reputations are established, sellers can charge higher prices through which they are compensated for their initial investment in reputation.

Reputation systems provided by online market platforms, thus, supply a sanctioning infrastructure that incentivizes cooperation (Yamagishi 1986; Kollock 1998). This solves the first-order cooperation problem—the trust problem—arising between sellers and buyers at the transaction stage. Reputation systems thus supplant the role of sellers’ moral norms (i.e. their consideration for the buyers’ outcomes) in solving the trust problem. Yet, to effectively incentivize cooperation, reputation systems rely on traders’ feedback on their experiences with other traders. The higher the rate of truthful feedback on completed transactions in a market, the sooner are untrustworthy traders detected, which reduces their incentives to enter the market in the first place (Przepiorka 2013). Conversely, a too low rate of truthful feedback can lead online markets to deteriorate (Akerlof 1970; Janssen 2006).

Because leaving feedback is costly in terms of time and effort and is usually not in itself sanctioned by online market platforms, maintaining a sufficient level of information about traders’ past behaviors presents a collective goods problem (Kollock and Smith 1996; Bolton, Katok, and Ockenfels 2004; Diekmann et al., 2014; Tadelis 2016). This is a second-order cooperation problem that refers to the question of who rewards cooperative sellers and punishes noncooperative ones by feeding feedback information into the reputation system (Heckathorn 1989; Kollock 1998). Yet, high feedback rates have been observed in both online markets for legal (Resnick and Zeckhauser 2002; Dellarocas, Fan, and Wood 2004; Diekmann et al., 2014) and illegal goods and services (Kruithof et al., 2016). Why do online traders leave feedback after completed transactions? How can (second-order) cooperation at the feedback stage be explained? We hypothesize that moral norms play an essential role in sustaining market action by solving the collective goods problem arising at the feedback stage in online markets.

2.2 The moral embeddedness of markets

The role of morality in markets has been extensively debated in sociology and economics (Hirschman 1982; Platteau 1994; Gintis et al., 2005). The line of research investigating the driving force behind markets and market action has relied on two distinct notions of morality in markets (Hitlin and Vaisey 2013). First, the question of what is moral concerns people’s understanding of the relative appropriateness of actions and beliefs. Economic sociologists have used this concept to explain how moral norms that stipulate the appropriateness of certain actions have had a bearing on the emergence of markets for particular commodities and services (e.g. life insurance or human organs) (Zelizer 1978; Beckert 2006; Fligstein and Dauter 2007; Fitzmaurice et al., 2020). Even cryptomarkets for illegal goods regulate trade in certain goods on moral grounds, as is the case with, for instance, restrictions on child sexual abuse material (Morselli et al., 2017).

Second, what is moral refers to universal concerns of justice, fairness, and harm—equating moral with prosocial, altruistic, and other-regarding values (Greene 2013). This concept of morality has helped explain how cooperative exchanges come about in existing markets. In this sense, moral norms are defined as other-regarding preferences that are followed by individuals even if doing so implies incurring personal costs (Elster 1985; Bicchieri 2006). It has been argued that moral norms enter cooperative market exchanges via market participants’ considerations for their trading partners’ outcomes (Beckert 2006; Sandberg 2012). Moral norms can thus reduce transaction costs through their capacity to produce trust (Uslaner 2002) and trustworthiness (Gambetta and Przepiorka 2014), especially when legal, relational, and organizational assurances are absent or less effective (Granovetter 1985; Platteau 1994). In this article, we build on this second notion of morality as a driving force of information exchange facilitating market action.

Previous research has hinted at the capability of moral norms to facilitate local information exchange supporting individual drug users’ decisions in conventional illegal markets (Bourgois 1998; Karandinos et al., 2014) or drive some cryptomarket buyers to generate information on the quality of goods traded in the market (Bancroft 2020). Here we suggest that moral norms play a much more essential role: they allow for the emergence of mutually beneficial economic exchanges in cryptomarkets by motivating the provision of information to reputation systems at large. Next, we devise a theoretical framework that allows us to contextualize the role or moral norms in the broader set of motives for leaving feedback on completed market transactions.

2.3 Motives for leaving feedback

We synthesize the insights from different strands of literature referred to in the introduction and subsume different motives for giving feedback under four broad terms borrowed from the behavioral and experimental social sciences (e.g. Bowles and Gintis 2011): self-regarding motives, other-regarding motives, reciprocity, and conformity. Below and in Table 1 (first two columns), we describe these concepts in abstract terms and give corresponding examples of their manifestations in the context of reputation-based online markets in general. Table 1 further lists motive cues we use to determine the presence of particular motives in the feedback texts that were posted in the cryptomarkets we study (third column). In the text that follows, numbers in parentheses correspond to motive manifestations and motive cues listed in Table 1. However, we describe motive cues in more detail in the next section.

Table 1.

Motive cue development based on our motive framework.

Motive categories		Motive manifestations	Motive cue
Self- regarding	Intrinsic	1. Obtain material rewards from platform or trading partner for leaving feedback 2. Obtain positive feedback from the trading partner after providing positive feedback to them 3. Obtain benefits from repeated interactions with trading partner 4. Receive assistance from platform or trading partner 5. Attain status via helpful or informative feedback 6. Feel better from contributing to the reputation system 7. Feel better from expressing feelings triggered by experience	1. NA: Platforms and sellers do not directly incentivize buyers to leave feedback 2. NA: Platforms only allow one-sided feedback from buyers 3. Cue: Intent to purchase again from same seller a 4. Cue: Reach out to seller or platform to demand an action b 5. NA: Status attainment not possible for buyers because feedback is anonymized 6. Cue: Share objective facts about the experience c 7. Cue: Express feelings about the experience
	Extrinsic	8. Seek approval from others by adhering to feedback-giving norms	8. NA: Platforms and sellers cannot directly sanction buyers’ feedback behavior
Other- regarding	Intrinsic	9. Benefit others irrespective of their role in current transaction (either seller or other buyers)	9. Measure: Avoid harming seller with a bad rating while elaborating on a subpar experience in feedback text to benefit other buyers Cue: co-occurrence of cues 10 and 12 Cue: co-occurrence of cues 11 and 12
	Extrinsic	10. Benefit trading partner based on previous experiences with them 11. Avoid harming trading partner despite a subpar experience 12. Benefit others contingent on their role by informing them about the experience	10. Cue: Help seller for reasons unrelated to the current experience (e.g. based on previous experiences with same seller) d 11. Cue: Avoid harming seller despite a subpar experience 12. Cue: Help other buyers by informing them about the experience
Reciprocity	Direct	13. Benefit/harm trading partner for received feedback 14. Benefit/harm trading partner for experience	13. NA: Platforms only allow one-sided feedback from buyers 14. Cue: Benefit/harm seller for the experience (incl. for extras and free goods received)
	Indirect upstream	15. Benefit trading partner for received feedback from other trading partners 16. Benefit next buyer for received feedback from other buyers in the past	15. NA: Platforms only allow one-sided feedback from buyers 16. Unlikely to be explicitly revealed in text; implicitly in other cues
	Indirect downstream	17. Benefit trading partner for providing feedback to their other trading partners	17. NA: Platforms only allow one-sided feedback from buyers
Conformity	Self-regarding intrinsic	18. Feel better from conforming to feedback left on trading partner by others 19. Conform to others’ feedback when uncertain about experience or how to leave feedback	18. Unlikely to be explicitly revealed in text; implicitly in cues pertaining to self-regarding intrinsic motives 19. Unlikely to be explicitly revealed in text; implicitly in cues pertaining to self-regarding intrinsic motives
	Self-regarding extrinsic	20. Seek approval from others by conforming to feedback left on trading partner by others	20. NA: Platforms and sellers cannot directly sanction buyers’ feedback behavior

a

This cue emerged during manual coding. We identify it using automatic word matching of specific phrases (e.g. “will be back”).

b

Reaching out to the platform was a separate coding category that we eventually merged with reaching out to the seller as it hardly occurred during manual coding.

c

Sharing knowledge beyond relevant information about the transaction was a separate coding category that we eventually dropped as it hardly occurred during manual coding.

d

This cue can also capture strategic intentions of establishing repeated interactions with the same seller.

Self-regarding motives refer to people’s desire to maximize personal material and psychological benefits and minimize costs. Self-regarding motives are intrinsic, if one’s actions produce these material (1–4) or psychological (5–7) benefits or costs (Andreoni 1990). Self-regarding motives are extrinsic if the social consequences of one’s actions produce these benefits and suppress these costs. Extrinsic self-regarding motives (8) are thus equivalent to social norms (Coleman 1990; Bicchieri 2006; Horne and Mollborn 2020), partly overlap with a broader notion of conformity (see below), but must be kept distinct from group solidarity (Brewer and Miller 1996), which falls under other-regarding motives.

Other-regarding motives refer to people’s considerations for others’ material and/or psychological benefits and costs. Other-regarding motives can thus be seen as extensions of intrinsic, self-regarding motives, which come into play when people assign non-zero weights to others’ outcomes (Messick and McClintock 1968; Becker 1976; Fehr and Schmidt 1999; Van Lange 1999). Other-regarding motives are intrinsic if they affect behavior independent of information about the person whose outcomes are taken into consideration (9). Other-regarding motives are extrinsic if they affect behavior contingent on information about the person whose outcomes are taken into consideration (aka group solidarity) (10, 11, and 12). Since both intrinsic and extrinsic other-regarding motives do not depend on the expectations of others, nor these others’ sanctioning threats, we equate them with moral norms (Bicchieri 2006). From here on, we will refer to moral norms as other-regarding motives.

Reciprocity refers to people’s inclination to reward fair and punish unfair behavior at a cost to themselves (Fehr, Fischbacher, and Gächter 2002). An action triggered by reciprocity is a reaction that aims at having a consequence for the “reciprocatee” (i.e. the actor to whom the reciprocating act pertains) and, as such, can be treated separately from other-regarding motives and social norms (Gouldner 1960; Falk and Fischbacher 2006). Reciprocity is direct if one rewards or punishes the reciprocatee for their previous actions toward oneself (13–14). Reciprocity is indirect if one rewards or punishes the reciprocatee for the actions of a third party toward oneself (upstream, 15–16) or for the actions of the reciprocatee toward a third party (downstream, 17) (Simpson et al., 2018).

Conformity refers to people’s desire to align their opinions and behaviors with others (Cialdini and Goldstein 2004) and can be conceived as intrinsic (18–19) or extrinsic (20) self-regarding motive (see above).

3. Identifying motives for leaving feedback from feedback texts

In this section, we develop our theoretical framework into a coding scheme mapping individual motives for writing feedback in feedback texts. We rely on Vaisey’s (2009) dual-process model that integrates two views on the question whether statements individuals make can reflect their true internal reasons for action (Campbell 1996). On the one hand, when motives are explicitly prompted by others, individuals engage in conscious deliberation to justify their actions in a socially acceptable manner. This is in line with Mills’ “vocabularies of motives”: typal vocabularies deemed appropriate for motive verbalization in certain social contexts (Mills 1940). On the other hand, when faced with everyday tasks, individuals make automatic decisions that reflect and communicate their internal values (Vaisey 2009). In line with the latter view, we argue that, apart from being reflective of individuals’ different experiences, individuals’ decisions on how to phrase feedback in the cryptomarket context are quick, automatic, and well-aligned with their internal values.

By browsing through online markets, market participants learn “vocabularies of feedback”: socially acceptable ways of expressing one’s feedback. Our decision to treat choices between these different “vocabularies of feedback” as an automatic rather than a deliberative decision is informed by our knowledge about the reputation systems of the cryptomarkets we study and our thorough reading of discussions in market-affiliated online forums. The social context of feedback writing in online markets limits the relevance of deliberative justifications of one’s actions for several reasons. First, while sometimes prompted in forums and FAQ pages, textual feedback sharing is voluntary and barred from having major repercussions on the individual.⁴ Second, in highly anonymized illegal online markets, individuals feel less compelled to act in accordance with their “real-life” social roles and identities. In the markets we study, usernames (i.e. pseudonyms such as “FlyHigh”) are additionally anonymized when feedback is displayed—which further encourages truthful reporting as tracing feedback to the original author’s username is difficult.

Third, even if usernames were not anonymized, in cryptomarkets as large as the ones we study, the risks of retaliation for honest feedback are limited; if the negatively rated seller would decide to not engage in future trades with the buyer, the buyer could easily purchase from alternative sellers instead.⁵ Further, marketplace forums confirm that giving feedback was a routine act, whereas deliberative discussions of market experiences were delegated to forums. In such a context vocabularies of feedback very likely capture the expressive intentions of their authors (Markoff, Shapiro, and Weitman 1975). Next, we describe how we translate the possible manifestations of three of our four main motive categories (columns 1 and 2 in Table 1) into text coding categories (column 3 in Table 1).

First, our decisions on which motives could manifest in the three cryptomarkets were determined by the design of these markets. All of the markets we study featured reputation systems relying on buyer-provided information.⁶ The seller would first post an item listing including the product description and price. The buyer would then decide whether to buy the listed product and initiate the payment. Finally, buyers could leave feedback after the seller reported the item as shipped. This feedback consisted of a quantitative rating (on a 3- or 5-point scale) and an optional textual description (textual feedback). Neither of the markets we study (see next section and the Appendix section) included explicit (sanctions or) rewards for (not) leaving feedback, which is why we do not consider cues denoting intentions to obtain benefits in return for leaving the feedback (1, 8, and 20)⁷. As sellers did not leave feedback on buyers, we disregard cues pertaining to behaviors conditioned on (previously) receiving feedback (2, 13, 15, and 17). Similarly, we exclude attaining status through feedback giving (5) as feedback could not be linked back to the usernames of buyers who wrote it.

Second, using an iterative process, we consulted a sample of random feedback texts to establish which motives included in our theory framework could hardly be measured with our data.⁸ In this process, we selected available cues and adjusted the coding instructions to better reflect the characteristics of feedback texts from our markets.⁹ This allowed us to translate our theory-rooted motive manifestations into a coding scheme capturing cues in our textual data (Grimmer, Roberts, and Stewart 2022). Through this process, we defined nine measures of motives in feedback texts. Eight measures are based on textual motive cues. Four cues pertain to intrinsic self-regarding motives: the intent to purchase from the seller in the future (3, e.g. “all great, will order again”), attempts to receive assistance with one’s order (4, e.g. “Refund money please”), sharing information about the order (6, e.g. “Arrived quickly, good product”), and expressing emotions about the order (7, e.g. “AMAZING!!!!,” but also “Absolutely appalling”). Three cues map extrinsic other-regarding motives: benefitting the trading partner based on longer-standing interactions (10, e.g. “great service as always!”), avoiding harming the partner’s reputation in spite of a subpar experience (11, e.g. “Didn’t get the package, I’m not sure it’s his fault though.”), and benefitting other buyers by informing them about the experience (12, e.g. “Flawless trade, I highly recommend,” but also “plz don’t buy he is scamer [sic] plz trust me guys”). Finally, we define one cue pertaining to reciprocity, capturing the intention to benefit or harm the seller in exchange for the experience (14, e.g. “Pretty nice, thank you”). In addition to these eight cue-based measures, we base one measure (item 9 in Table 1) on two analyses: (1) the co-occurrences of several other-regarding motive cues; and (2) the discrepancy between the quantitative rating assigned to a feedback and the polarity of the corresponding feedback text (i.e. the extent to which the text was positive or negative).¹⁰ Full coding instructions can be found in Supplementary Material, Section A.

Finally, forum discussions helped us guide these decisions and evaluate their validity in our cryptomarket contexts. For instance, forums indicate that overemotional feedback was not seen as serving the community (7, “…don’t you think … this market/forum have to be … cleaned from people who … are angry on [sic] … vendors…,” AlphaBay forum), that buyers should, at times, be mindful of sellers’ circumstances (11, “Sometimes stuff can happen, not the vendors [sic] fault …,” Silk Road forum), that recommendations and warnings were considered a form of consideration for the outcomes of other traders (12, “… thanks for your warning…it is helpful to the community…,” Silk Road forum), and that feedback was, indeed, used to reciprocate sellers’ services (14, “…if you approach someone in kindness it will be appreciated if you reciprocated …with … feedback,” AlphaBay forum).

Choices individuals make between vocabularies of feedback (i.e. motive cues in the third column in Table 1) indicate their motives for writing feedback (i.e. the first column in Table 1). For instance, a dissatisfied buyer that uses a vocabulary centered on venting pent up emotions (e.g. “Complete waste of time and money”) is more likely to be driven by self-regarding motives than one that emphasizes a concern for the safety of other buyers in the market (e.g. “Buyers beware. Ordered almost two weeks ago. Still have not received product”). Since people can have multiple motives for an action, we allow texts to be coded for multiple motive cues. However, some texts reveal little beyond a short description of the transaction (e.g. “good product” or “never arrived”) and do not contain clear indications of any motive cues. In such cases, our coders had to find a balance between capturing weak indications of motives and avoiding speculation.¹¹

4. Data and methods

4.1 Data

We surveyed three cryptomarkets: Silk Road (operational: February 2011 to October 2013), AlphaBay (operational: December 2014 to July 2017), and Hansa (operational: July 2015 to July 2017) (Kruithof et al., 2016), all of which offered a range of illegal goods—mainly drugs, stolen personal and financial information, and weapons. We use Silk Road data collected by Christin (2013), AlphaBay data collected by Norbutas, Ruiter, and Corten (2020), and Hansa data collected by Lewis (2016) by automatically browsing and downloading snapshots of marketplace webpages.¹² The data contain quantitative and textual feedback information, seller usernames, item IDs, prices, and categories.

We start by evaluating buyers’ propensities to leave feedback in the three cryptomarkets. Table 2 provides an overview of feedback rates across markets, along with the shares of positive, neutral, and negative quantitative ratings. In Silk Road, feedback was needed for the finalization of the order, which implies a 100 per cent feedback rate. In addition, we find that 98 per cent of Silk Road feedback contained some additional text. In Hansa, feedback was also given upon finalization; yet, only 62 per cent of feedback contained additional text. In AlphaBay, we find that buyers left feedback on around 51 per cent of transactions, 81 per cent of which also included a textual description (see Supplementary Material Section B for details). In all three markets, more than 96 per cent of feedback ratings are positive. This is consistent with previous accounts from legal markets (Bolton, Greiner, and Ockenfels 2013; Diekmann et al., 2014).

Table 2.

Feedback provision rates and quantitative feedback shares and counts per market.

Feedback	Silk Road	AlphaBay	Hansa
Overall rate	100%	51%	100%
(N = )	(184,751)	(2,361,314)	(42,843)
Positive	97%	96%	97%
Neutral	2%	1%	2%
Negative	1%	3%	1%
With text	98%	81%	62%
(N = )	(181,293)	(1,921,600)	(26,597)
With English text	96%	72%	60%
(N = )	(178,104)	(1,703,888)	(25,738)

a

In Silk Road, buyers could choose a rating on a scale from 1 (the worst) to 5 (the best). We recode these ratings as follows: (1)—Negative; (2, 3, and 4)—Neutral; (5)—Positive.

Our full dataset contains 2,588,908 ratings from three markets; after excluding 459,418 ratings that were not accompanied by any additional text and another 221,760 accompanied by texts that contained only symbols or were written in languages other than English, our final dataset comprises 1,907,730 feedback texts from three markets. We provide a detailed breakdown per market in Table 2. These texts are short: on average, they contain the most words in Silk Road (M = 12.7, SD = 16.7), followed by AlphaBay (M = 10, SD = 9.3), and Hansa (M = 9.7, SD = 12.7).

4.2 Text analysis

To infer motives for writing feedback in cryptomarkets, we code feedback texts for different motive cues (as per Table 1 and the procedure described in Section 3) and evaluate whether the text expresses a positive, neutral, or negative evaluation (text polarity as a part of the measure 9 in Table 1). Our data, however, contained too many observations for the manual coding. We addressed this by expanding manual coding of a sample of texts onto the remainder of our dataset using text mining.

We began by sampling a subset of texts for manual coding. We oversampled less frequent neutral and negative feedback texts, as well as texts from smaller sellers (see Supplementary Material, Section C for details), to obtain a balanced sample for our text mining algorithm (Japkowicz 2000). The final sample consisted of 2,000 texts (of which 1,000 accompanied positive ratings, 333 neutral ratings, and 667 negative ratings), each independently coded by three trained coders who followed explicit instructions for each coding category (Krippendorff 2004). We measure intercoder agreement on each cue using the prevalence- and bias-adjusted Fleiss’ kappa—PABAK (Sim and Wright 2005; Klein 2018). With PABAK scores ranging between 0.42 and 0.9, we observe moderate to substantial agreement between coders across the surveyed cues (Landis and Koch 1977). These agreement rates are comparable to other studies working with similar coding tasks (Liu et al., 2012; Hoover et al., 2020).¹³ We then code each text for the categories that were agreed on by at least two out of three coders.

We extend our coding onto the remaining feedback texts using supervised machine learning in conjunction with transformer neural network language models (Vaswani et al., 2018; Wolf et al., 2020). Transformer models employ self-supervised learning to represent words as vectors in a high-dimensional space, grouping contextually similar words together (Vaswani et al., 2018; see Supplementary Material, Section D for more details). Transformer models excel in capturing both the immediate word context (e.g. the word “dog” in proximity to other canine-related terms) and the varied contexts in which words appear (e.g. a dog’s “bark” versus tree “bark”). We use RoBERTa transformer model trained on several large textual corpora (see Supplementary Material, Section D) (Devlin et al., 2019; Liu et al., 2019).

We then use sequential transfer learning, fine-tuning RoBERTa model to our particular task (Do, Ollion, and Shen 2022). During the fine-tuning process, the model learns to associate word occurrences in texts with coding patterns in manually coded data (Sun et al., 2019). The model then assigns varying weights to word vectors based on their significance for coding for each category. Finally, the weights are converted to probabilities denoting how likely it is that a text belongs to each one of the relevant categories. We fine-tune RoBERTa model using a training set containing 80 per cent of our manually coded texts and evaluate its performance against human coders using the remaining 20 per cent as the test dataset. RoBERTa achieves an accuracy of 0.83 on the text polarity coding task and an accuracy of 0.91 with an average F-score of 0.77 on our motive cues.¹⁴

5. Results

5.1 Motives for leaving feedback

We first discuss the overall distribution of motives in feedback texts, thus contextualizing the role of other-regarding motives (i.e. moral norms) in solving the (second-order) cooperation problem. In Fig. 1, we show the shares of all motive cues across all feedback texts, with other-regarding motives combined into a single category. We provide a more detailed analysis of the different types of other-regarding motives in the next subsection. For a better overview, we show results in Fig. 1 for feedback with positive (green), neutral (orange), and negative (red) ratings and for the three markets separately.¹⁵

Figure 1.

Shares of motive cue occurrences per quantitative feedback rating across three cryptomarkets.

We find that, in total (blue), around 21 per cent of feedback texts were driven by other-regarding motives in all three markets.¹⁶ This percentage is even higher when it comes to feedback with neutral and negative ratings: up to 56 per cent of neutrally and 31 per cent of negatively rated feedback indicates the presence of other-regarding motives.

At least one cue indicating self-regarding motives appears in up to 79 per cent of feedback texts across the three markets.¹⁷ The most frequently occurring cue captures fact sharing (6) and appears in between 56 and 62 per cent of all feedback texts. This cue is particularly frequent in neutrally and negatively rated feedback, indicating that subpar experiences are more often described in detail. Between 24 and 32 per cent of feedback texts contain the cue of emotional expression (7). Around 8 per cent of texts indicate the willingness to engage in future trades with the seller (3), while only about 1 per cent tries to draw attention to a problem with the transaction (4), except in the Silk Road market.¹⁸ Between 19 per cent and 20 per cent of feedback texts across the markets contain the cue indicating direct reciprocity (14), mostly in positively rated feedback—negative reciprocity is rather rare.

Finally, some texts were not coded for any motive cues. Their percentage ranges from 14 in Silk Road to 22 in Hansa. Such texts rarely provide insightful factual information (e.g. “Perfect,” “Good,” “A++,” “nothing,” “scam”) and may be motivated by buyers’ intention to acknowledge the arrival of the product, share the fact that the product was not delivered, or contribute at least some rudimentary information to the reputation system. Taken together, these findings provide a fine-grained picture of how important different psychological mechanisms are in supporting the reputation systems of online markets.

5.2 Other-regarding motives decomposed

In order to better understand the role of other-regarding motives in overcoming the collective good problem at the feedback stage in online markets, we explore these motives in more detail. Figure 2 shows the shares of the different types of other-regarding motives across the three markets.

Figure 2.

Shares of other-regarding motive occurrences per quantitative feedback rating across three cryptomarkets

In all three markets, between 10 per cent and 11 per cent of all feedback texts contain one of the cues indicating extrinsic other-regarding motives (cues 10, 11, or 12). This can be seen in Fig. 2 by adding up the percentage of “All” occurrences of cues 10, 11, and 12 (e.g. Hansa: 2.6% + 2.3% + 5.3% = 10.2%). The cue indicating the intention to benefit the trading partner (10) occurs in 2–4 per cent of texts, and the cue capturing reluctance to harm the trading partner’s reputation (11) occurs in 2–3 per cent of feedback texts. The former mostly occurs in feedback texts with positive ratings, while the latter is more frequent in feedback with neutral or negative ratings. The cue capturing other-regarding motives oriented toward other buyers in the market (12) appears in 5 per cent of feedback texts across markets; the prevalence of this cue is especially high in texts with negative ratings, indicating its importance in flagging bad experiences and untrustworthy sellers.

We further evaluate intrinsic other-regarding motives (9). We find that between 10 and 11 per cent of texts across the three markets contain indications of intrinsic other-regarding motives.¹⁹ These motives are most prevalent in texts with neutral ratings. Texts with cues of intrinsic other-regardig motives balance between the dishonesty of leaving a positive rating after a bad experience and the damage a seller could incur from a negative rating.

5.3 Other-regarding motives in challenging market situations

Going beyond a merely descriptive account of the motivational landscape of reputation-based cryptomarkets, we explore the role other-regarding motives play in two critical market situations: the evaluation of new sellers who have just entered the market and the exclusion of untrustworthy sellers from the market. In the former case, buyers need to take a risk when purchasing from a new seller without a reputation. The riskiness of this situation might induce more feedback motivated by other-regarding preferences aimed at helping the newbie seller (Diekmann et al., 2014) or sharing valuable insights on the new seller with the rest of the buyers. In the latter case, a suspicion that a seller is abusing trust can activate other-regarding preferences that help warn other buyers—and, thereby, alarm the market administration—about the risks of trading with a fraudulent seller. In this section, we analyze data from AlphaBay, the only market in which we have information on sellers who were excluded from the market (i.e. banned) by the market administration. In Fig. 3a, we plot the weekly shares of differently motivated feedback received by 4,242 new sellers that stayed in the market for at least 10 weeks. In Fig. 3b, we plot the feedback shares of eighty-six sellers that survived in the market for at least 10 weeks before being banned.

Figure 3.

Shares of motive cue occurrences in the first and last 10 weeks of seller existence in AlphaBay. (a) 10 weeks before market entry and (b) 10 weeks before market ban.

In Fig. 3a, we see that the share of extrinsic other-regarding motives directed toward helping other buyers (12) is slightly higher in the week of new sellers’ market entry than in the later weeks. However, this pattern is even more pronounced for other cues—including factual information sharing (6), self-regarding motives (3, 4, and 7), and reciprocity (14)—the share of which also decreases over time, giving way to motives of helping the seller (10), avoiding harming the seller (11), and feedback texts without any motive cues. This analysis does not provide clear evidence that moral norms play a particularly important role in identifying trustworthy sellers among new market entrants compared to other motives.

In Fig. 3b, we show that the share of other-regarding motives directed at other buyers (12) starts increasing as early as five weeks before a seller is banned, constituting up to 20 per cent of total feedback in the last week of the seller’s existence. At the same time, seller-oriented extrinsic (10 and 11) other-regarding norms decrease in their share, along with a drop in the share of feedback motivated by the sharing of factual information, other self-regarding motives, and reciprocity.²⁰ Moral norms are thus particularly important for supporting the functioning of reputation system when it comes to warning the buyer community and market administration about untrustworthy sellers.

6. Discussion and conclusion

Reputation systems are ubiquitous in today’s online markets for social and economic exchange because they have been proven to be effective in establishing trust between anonymous traders (Bolton, Greiner, and Ockenfels 2013; Jiao, Przepiorka, and Buskens 2021). However, the capability of reputation systems to establish trust is only as good as the feedback information shared by traders about completed exchanges. If truthful feedback is provided at lower rates, untrustworthy traders will have a higher incentive to enter the market because they will remain undetected for longer. This will, in turn, increase uncertainty and information costs, and may cause the market to deteriorate (Akerlof 1970; Shapiro 1983; Janssen 2006). It is therefore crucial for our understanding of reputation-based online markets to know why traders leave feedback after completed exchanges. Previous research has addressed this question by means of surveys with convenience samples or digital trace data on feedback-giving behavior obtained from online market platforms (Hennig-Thurau et al., 2004; Diekmann et al., 2014). However, despite these efforts, we have little knowledge on how frequent different motives for leaving feedback occur and co-occur in real-world online markets, and how these different motives promote mutually beneficial market exchange.

We provide answers by means of analyzing two million feedback texts left, alongside numerical ratings, by anonymous traders in three large cryptomarkets for illegal goods. We chose to address this question in an illegal market context because we wanted to measure the motivational landscape of reputation-based online markets unconflated by legal and relational assurances of cooperative market exchange (Przepiorka, Norbutas, and Corten 2017). Moreover, in line with the assertions that a generalized morality would be needed for cooperative market exchange in the absence of legal and relational assurances (Granovetter 1985; Platteau 1994), we sought evidence of whether moral norms are indeed an essential driver of feedback provision in illegal online markets.

Research has highlighted how moral norms promote cooperation in market contexts where friendship and kinship ties sustain moral obligations (Greif 1989; Bourgois 1998; Sandberg 2012; Karandinos et al., 2014). Yet, traders in large anonymous online markets make decisions on whom to trust based on the information in reputation systems, rather than relying on personal ties or expectations of other’s good intentions (Wehinger 2011; Przepiorka, Norbutas, and Corten 2017; Duxbury and Haynie 2021). Despite this, we argue that the role of moral norms does not vanish in online markets; rather, it shifts to supporting the exchange of information crucial to the functioning of reputation systems.

We demonstrate the importance of moral norms—alongside other motives—in online markets using the evidence from our text analysis. To infer different motives from feedback texts, we build on frameworks detailing how different motives and norms drive individual actions (Vaisey 2009) and affect the choice of textual expression (Campbell 1996). Bearing in mind the particularities of the cryptomarkets we study, we develop a coding scheme based on our comprehensive theoretical framework on motives for leaving feedback. We then combine manual text analysis and text mining methods to infer motives behind the two million feedback texts from the cryptomarkets Silk Road, AlphaBay, and Hansa.

First, our results corroborate that traders successfully overcome the second-order cooperation dilemma at the feedback stage in cryptomarkets by leaving feedback at high rates (albeit at lower rates than reported in legal online markets) (Resnick and Zeckhauser 2002; Bolton, Greiner, and Ockenfels 2013; Diekmann et al., 2014). Second, we provide a first estimate of the share of feedback in online markets motivated by moral norms. We find that moral norms motivate around 21 per cent of all feedback with text, and this result remains robust across the three cryptomarkets we survey. Third, we show that moral norms drive a very high share of neutral and negative feedback evaluations overall. Further, we observe an increase in the share of feedback motivated by moral norms—coupled with a decrease in shares of other motives—in the weeks preceding the exclusion of fraudulent traders from the market. What can these results tell us about the importance of moral norms in sustaining cooperation in online markets in general?

First, by driving a substantial proportion of feedback, moral norms of cryptomarket traders help lower transaction costs (Przepiorka 2013) and prevent the breakdown of cooperation in online markets (Janssen 2006). That is, the higher the rate of truthful feedback, the quicker will untrustworthy traders be screened and the less likely will they be encountered by buyers. In a market with mostly trustworthy traders, buyers will, thus, pay less for traders’ reputations, which will reduce transaction costs (Jiao, Przepiorka, and Buskens 2021). Combined with Janssen’s (2006) theoretical predictions, our results suggest that, in the absence of feedback motivated by moral norms, the benefits of trading in the market could decline by up to 20 per cent.²¹ Second, we observe a higher prevalence of moral norms in neutral and negative feedback and an increase in the share of morally motivated feedback in the last weeks of fraudulent sellers’ existence in the market. This suggests that moral norms drive an “early-warning system” assisting the community with the challenging and potentially morally loaded task (Stets and Carter 2012) of fraud detection. To explore the limits of online market survival in the absence of morally motivated feedback, future research could combine our results with computer simulations of different reputation systems.

However, we also find a substantial share of traders who face a moral dilemma when leaving feedback. After a negative trading experience, traders may wish to benefit other traders by leaving truthful feedback, while also striving to benefit or not harm their current trading partner. Consequently, inaccurate reporting can diminish the validity of reputation scores in measuring the trustworthiness of traders in the market (Tadelis 2016; Filippas, Horton, and Golden 2018). Exploration of market participants’ perceptions of the trustworthiness of feedback texts stemming from different moral concerns (Simpson, Harrell, and Willer 2013) and a deeper understanding of strategic avoidance of retaliation could shed more light on the role of such feedback in reputation systems.

We provide, to our knowledge, the first comprehensive evaluation of the relative importance of a range of motives participants in illegal cryptomarkets had for contributing information to the reputation system. While research so far relied on approaches that allowed to tie reciprocity to responding to feedback received from one’s trading partner (Resnick and Zeckhauser 2002; Jian, MacKie-Mason, and Resnick 2010), we are able to capture feedback written to reciprocate the transaction experience, finding that it is less prevalent than feedback motivated by moral norms. Reciprocal motives also sustain cooperative behaviors without enforcement through sanctions and can, thus, be seen as quasi-moral norms (Elster 2006). Further, a large share of feedback texts in cryptomarkets stems from participants’ self-regarding motives such as strategic considerations, seeking help, emotional release, or an intention to feel better.

Whereas previous research acknowledged that moral norms hamper market exchanges of goods and services considered immoral (Zelizer 1978; Beckert 2006), we explore how moral norms contribute to the establishing of cooperation in cryptomarkets that deal in goods and services that are often deemed immoral (e.g. drugs, stolen credit card data). Our results are largely consistent across three cryptomarkets that differ in size and longevity, indicating that our findings are generalizable to similar reputation-based online markets on the Dark Web. Research has deemed the possibility to engage in repeated interactions, attributability of actions to users, and the rewarding of information provision efforts to be the essential drivers of contributions to online communities (Kollock 1999). Here we show that moral norms and other motives support a system of generalized information exchange in reputation systems in which these drivers are absent. More broadly, thus, our results elucidate motives that underlie the information “gift economy” that supports cooperation in online communities and markets even under adverse conditions (Kollock and Smith 1996; Kollock 1999).

Our choice to code for explicit motive cues does limit deeper exploration of nuanced manifestations of different motives. At the same time, it makes our estimates of motive prevalence conservative. Ratings without text or with brief comments might stem from moral norms or a general positive feeling toward contributing to the cryptomarket community (Van Hout and Bingham 2013; Morselli et al., 2017; Bakken, Moeller, and Sandberg 2018; Bancroft 2020). Our approach cannot discern whether not leaving feedback altogether reflects time- and effort-saving concerns or a moral dilemma (where one protects the seller by not leaving bad feedback while also protecting buyers by not leaving good feedback either). Further development of our coding scheme complemented by other data analysis methods could shed more light on both the motives we surveyed and those we theorized about but were unable to identify.

Our survey of the motivational landscape of reputation-based online markets opens the door for a deeper understanding of how traders thread the fine line between engaging in rational self-interested transactions (Sandberg 2012; Pace 2017) and participating in setting community-level expectations about buyer and seller trustworthiness (Van Hout and Bingham 2013; Bilgrei 2018; Lorenzo-Dus and Di Cristofaro 2018; Masson and Bancroft 2018; Bancroft 2020; Sawicka, Rafanell, and Bancroft 2022). Further exploration of how vocabularies of feedback come about and are used across cryptomarkets could help better understand how cryptomarket cultures around feedback giving develop in response to information asymmetries between sellers and buyers.

Apart from consulting the reputation systems, market participants can assess seller quality by relying on past sales volumes, item listing quality, product samples, and seller-related discussions in affiliated marketplace forums (Tzanetakis et al., 2016; Moeller, Munksgaard, and Demant 2017; Bakken, Moeller, and Sandberg 2018; Ladegaard 2018; Lorenzo-Dus and Di Cristofaro 2018). To gain a more complete understanding of different modes of governance in these markets, future research could explore how moral norms and organizational assurances support valuation and sanctioning by buyers and market administrators.

Finally, our approach demonstrates the utility of advanced text mining algorithms in extending the manual coding of a large number of texts (Nelson et al., 2018). Future research could use our method, coding millions of feedback texts for individual internal states, to assess how motives vary by author and target characteristics or transaction features. Evaluating the changes in motives as markets evolve and grow—or are disturbed (Ladegaard 2020; Norbutas, Ruiter, and Corten 2020)—would help us better understand the interplay of organizational assurances, actions of market administrators, individual values, and the creation of norms and expectations surrounding the maintenance of cooperative exchanges in cryptomarkets. Additionally, one could explore whether different markets and market spheres receive differently motivated information based on their organizational features. This could yield important insights into the interplay of organizational and psychological mechanisms in supporting cooperation in different market contexts.

Ethical approval statement

This study was approved by the Ethics Committee of the Faculty of Social and Behavioral Sciences of Utrecht University (approval no. FETC20-351, 21-059).

Footnotes

1

We follow Coleman’s definition of situations that involve trust as those where “the risk one takes depends on the performance of another actor” (Coleman 1990: 91). We define trustworthiness as the sellers’ intention to deliver goods or services to the buyer as promised. Trust is the buyer’s belief about the seller’s trustworthiness and/or their ability to deliver goods or services as promised; based on these beliefs, the buyer decides whether to engage in a transaction with the seller (Przepiorka 2023).

2

A functioning legal system can promote cooperation in extra-legal contexts if the revelation of previously shared misdeeds can be expected to trigger the legal punishment of the perpetrators (Gambetta and Przepiorka 2019).

3

Other organizational assurances in cryptomarkets include payment protection mechanisms (e.g. escrow service) and enforcement by market administrators. Using the payment protection options is not always mandatory and can easily be (and often is) circumvented by sellers (Andrei et al., 2023). Further, market administrators have the power to regulate exchanges in the market and exclude untrustworthy traders; yet, they tend to encourage direct conflict resolution and intervene only in cases of extreme fraudulence (Morselli et al., 2017). Moreover, market administrators have a vested interest in maximizing the number of transactions (regardless of seller trustworthiness) and are able to shut down the whole platform pocketing the funds still in the payment protection system (Moeller, Munksgaard, and Demant 2017; Ladegaard 2020; Norbutas, Ruiter, and Corten 2020).

4

While we do not expect this to be common, we cannot exclude that some individuals felt pressured to express their experience in a certain (less automatic or honest) manner when prompted to rate their transaction by the reputation system.

5

There is also a possibility that some buyers’ feedback reflected fear of retaliation in the form of being doxed (having one’s real-life identity publicly revealed). However, as buyers could avoid giving feedback altogether, trade with other sellers or make new accounts to circumvent seller-imposed trade restrictions, we do not consider these concerns to be a major factor in our analysis. What is more, using one’s real identity (e.g. name or postal address) was discouraged in cryptomarkets and doxing was strictly sanctioned by market administrators.

6

We provide a more detailed description of other market features not essential for our coding scheme development in the Supplementary Material.

7

While this was not systematically integrated into the market, some sellers did provide samples or extra goods in their shipments expecting good feedback in return (Ladegaard 2018). We provide some insights on the impact of such practices in Supplementary Material, section F.

8

For instance, we concluded that cues 16, 18, and 19 were unlikely to be identifiable from texts only. Further, we initially conceived one more motive cue related to the motive manifestation of feeling better from contributing to the reputation system. This cue captured feeling better from sharing more extensive expert knowledge with the community (e.g. by providing advice on how to dose a certain drug). Yet, we dropped this cue after surveying sample texts, as it became obvious it was difficult to distinguish from sharing factual information about the experience in our data (current cue 6).

9

We initially considered receiving assistance from the market platform or the trading partner (motive manifestation and cue 4) as two separate cues, but concluded that the former was so infrequent in our data that it did not warrant a cue of its own. Similarly, whereas we did not expect to be able to identify cue 3 from individual texts only, our coders pointed out that obtaining benefits from repeated interactions was captured in texts discussing an intention to purchase from the same seller in the future.

10

After a subpar transaction, one might wish to avoid damaging the seller’s reputation (11), while still wishing to benefit others in the market by appropriately informing them about the experience (12). One can address these conflicting considerations either explicitly in text (9: co-occurrence of cues 11 and 12); or implicitly (9: measure). The latter implies leaving a more positive quantitative rating—since these ratings enter seller’s formal reputations—while reporting a less positive experience in the feedback text, thus informing other buyers truthfully (Filippas, Horton, and Golden 2018)—as texts have no effect on the formal reputation score.

11

We instructed coders to focus on information explicitly revealed in texts and discouraged them from relying on subjective interpretations when coding ambiguous texts. Still, we kept the coding instructions short and let coders resolve dillemas without our direct input to prevent inflation of intercoder agreement (Hoover et al., 2020).

12

These data provide comprehensive coverage of markets without indications of systematic missingness. Silk Road data contains a gap of 19 days due to website maintenance or collection issues (Christin 2013); AlphaBay data covers approximately 85% of all transactions on the website; Hansa data were collected in a manner that should grant comprehensive coverage (Lewis 2016). We provide more details on data coverage in Section B of the Supplementary Material.

13

We provide more details on the coding procedure and list PABAK scores in the Supplementary Material, Section A.

14

F-score captures a method’s ability to identify true positives and avoid false negatives (Aggarwal 2018). It ranges from 0 to 1, with values above 0.5 indicating satisfactory performance (Nelson et al., 2018). We discuss our model implementation, F-score averaging, and model performance per motive cue in Supplementary Material, Section D.

15

In Supplementary Material, Section F, we show the results on a subset of drug-related transactions to control for the heterogeneity of products that receive feedback; our results do not qualitatively change.

16

Around 3.3 per cent of texts in Silk Road, 1.7 in AlphaBay, and 2.8 in Hansa contain other-regarding motive cues without the presence of any other cues.

17

This number denotes the share of texts that contain at least one self-regarding cue and therefore cannot be inferred from Fig. 1 directly.

18

Cue (4) appears much more frequently in Silk Road. In Silk Road, the buyer could pay the seller in advance only after leaving feedback, so buyers would “hold their feedback hostage” by promising to update it once the goods arrive. This cue captures many such texts.

19

Silk Road has a higher share of such texts—also see note 18. As many of these texts did not reflect other-regarding motives, we have excluded them from all the analyses by filtering out texts that contained the cue “Reach out to the seller…” (4).

20

In Fig. F7 in Supplementary Material we also show how an increase in other-regarding feedback precedes and outnumbers the increase in other types of warnings about untrustworthy sellers.

21

We provide more details on this estimation in Supplementary Material, Section F.

Funding

This study was supported by Utrecht University, Faculty of Social and Behavioral Sciences, PhD fund.

Supplementary data

Supplementary data is available at SOCECO Journal online.

Conflict of interest statement

None declared.

Acknowledgements

We would like to thank Lukas Norbutas and Nicolas Christin for providing us with part of the data used in this research and their expert advice on aspects of data handling. We also thank Martin Abraham, Vincent Buskens, Diego Gambetta, Arnout van de Rijt, and the participants of the Cooperative Relations Lunch Seminar at the Department of Sociology of Utrecht University for their perceptive comments on earlier versions of this article.

Appendix

Details on Silk Road, Alpha Bay, and Hansa reputation systems

All three markets we survey featured reputation systems relying on buyer-provided information. Once the buyer decided to buy the listed product, the payment would be initiated and held up in “escrow”: a payment protection system where the funds are held by the market and only released to the seller once the buyer confirms receiving the order. This system was in place to protect buyers from fraud. However, in Silk Road and AlphaBay sellers could request buyers to “finalize early,” that is, release the funds before receiving the order and research shows that buyers in cryptomarkets preferred to rely on seller’s reputation scores rather than the escrow system (Andrei et al., 2023).

In all three markets, buyers could leave feedback after the seller reported the item as shipped. This feedback consisted of a quantitative rating (a three-point scale in AlphaBay and Hansa, and a 5-point scale in Silk Road) and a field for an optional textual description. In Silk Road buyers had to either confirm the arrival of goods or “finalize early” to have the funds released to the seller and the order closed (finalized). In case a buyer had failed to finalize within two weeks from shipping, the order was closed and automatically assigned the highest rating without any text. In Hansa, both the seller and the buyer had to finalize the order to close the transaction: if either side failed to finalize, the order would be canceled without feedback. Upon finalization in these two markets, the buyer would be prompted to leave (at least) quantitative feedback. In AlphaBay, leaving feedback was not mandated by the marketplace.

In all tree markets, quantitative ratings entered a seller’s reputation score, which was displayed next to the seller’s username. More detailed textual feedback per transaction was shown on the seller page (fifteen and ten most recent ratings in AlphaBay and Silk Road, respectively), together with corresponding quantitative ratings.