4
decentralized and incentivized federated learning which is
crucial to spreading the new generation of widely adopted
fair and trustworthy FL to the benefit of the data owner.
To the best of our knowledge, this paper is the first
systematic literature review on the topic of blockchain-
enabled decentralized FL with incentive mechanisms.
4 METHOD OF LITERATURE REVIEW
The goal of the following systematic literature review is
the identification of decentralized collaborative learning
solutions where participation is rewarded. The aim of this
study is not only to summarize all major publications, but
also to extend the research by having a guideline for current
and future practitioners.
Relevant publications are retrieved, filtered, and selected
by a methodical procedure. The procedure is inspired
by the Preferred Reporting Items for Systematic Reviews and
Meta-Analyses (PRISMA) methodology [27]. PRISMA helps
authors to improve the reporting of reviews. Within this
work, the PRISMA methodology is augmented with the
guide for information systems proposed by Okoli et al. and
Kitchenham et al. [28], [29]. The guidelines are a structured
approach to conduct a systematic literature review. They
consist of five core steps: (i) defining research questions,
(ii) searching for literature, (iii) screening, (iv) reviewing,
(v) selecting and documenting relevant publication and
extracting relevant information. The corresponding flow
diagram of the conducted SLR is illustrated in Fig. 1.
4.1 Research Questions
The review and selection of relevant literature is conducted
for extracting, preparing, and summarizing relevant
information for future researchers. The scope of relevant
information is described and limited by five corresponding
research questions.
RQ1 Overview: (i) What are possible applications of FLF?
(ii) What problems were solved? (iii) Across which
dimensions are the FLF papers heterogeneous?
RQ2 Blockchain: (i) What is the underlying blockchain
architecture? (ii) How is blockchain applied within
the FLF and what operations are performed? (iii) Is
scalability considered?
RQ3 Incentive mechanism: (i) How are incentive
mechanisms analyzed? (ii) How are the
contributions of workers measured?
RQ4 Federated learning: (i) Is the performance of
the framework reported? (ii) How comprehensive
are the experiments? (iii) Are non-IID scenarios
simulated? (iv) Are additional privacy methods
applied? (v) Is the framework robust against
malicious participants?
RQ5 Summary: What are the lessons learned from the
review?
4.2 Search Process
We conduct a well-defined search process of relevant
publications to allow for reproducibility. The main
keywords of interest are “Federated Learning”,
“Blockchain”, and “Game Theory”. Due to the existence
of synonyms (e.g. “Collaborative Learning” instead of
“Federated Learning”) and abbreviations (e.g. “DLT”
instead of “Blockchain”) the search term is extended
by taking these variations into account. The final case-
insensitive search term is the following:
Search Term
("Federated Learning" OR "Federated
Artificial Intelligence" OR
"Collaborative Learning") AND
("Blockchain" OR "Distributed Ledger"
OR "DLT") AND ("Mechanism Design" OR
"Incentive" OR "Reward" OR "Game Theory"
OR "game-theoretical" OR "Economics")
We selected 12 of the major computer science publication
databases. The search was conducted on November 21, 2021.
All search results retrieved at that date were taken as input
for further manual inspection. The current results of the
query can be obtained by following the hyperlink after each
database entry below:
• IEEE Xplore Digital Library (URL)
• SpringerLink Database (URL)
• ACM Digital Library (URL)
• ScienceDirect Database (URL)
• MDPI
2
(URL)
• Emerald insight (URL)
• Talor & Francis URL
• Hindawi (URL)
• SAGE (URL)
• Inderscience online (URL)
• Wiley (URL)
After applying the search term, we found
422 publications overall. In addition to this search, the
references were screened and 1 more eligible paper was
found and included for analysis.
Each publication that we retrieved based on our search
was exported or enriched by (i) document title, (ii) abstract,
(iii) author names, (iv) publication year, and (v) Digital
Object Identifier (DOI).
4.3 Selection Process
The eligibility of the literature corpus that we gathered
based on the string was evaluated independently by three
researchers. Duplicates were removed and several iterations
of manual verification were performed. First, we present the
criteria that we applied to include (or exclude) an article
for further consideration. Then, we discuss our manual
eligibility check.
4.3.1 Inclusion and exclusion criteria
The following three criteria must be fulfilled by an article in
order for it to be included for further analysis:
2. As MDPI does not allow for nested search terms, we tried
possible keyword combinations such as “Federated Learning” AND
“Blockchain” AND “Mechanism Design”