Näytä suppeat kuvailutiedot

Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces

Jorge Peña Queralta; Li Qingqing; Tomi Westerlund; Wenshuai Zhao

dc.contributor.authorJorge Peña Queralta
dc.contributor.authorLi Qingqing
dc.contributor.authorTomi Westerlund
dc.contributor.authorWenshuai Zhao
dc.date.accessioned2022-10-28T13:28:55Z
dc.date.available2022-10-28T13:28:55Z
dc.identifier.issn1877-0509
dc.identifier.urihttps://www.utupub.fi/handle/10024/165479
dc.description.abstract<p>The integration of edge computing in next-generation mobile networks is bringing low-latency and high-bandwidth ubiquitous connectivity to a myriad of cyber-physical systems. This will further boost the increasing intelligence that is being embedded at the edge in various types of autonomous systems, where collaborative machine learning has the potential to play a significant role. This paper discusses some of the challenges in multi-agent distributed deep reinforcement learning that can occur in the presence of byzantine or malfunctioning agents. As the simulation-to-reality gap gets bridged, the probability of malfunctions or errors must be taken into account. We show how wrong discrete actions can significantly affect the collaborative learning effort. In particular, we analyze the effect of having a fraction of agents that might perform the wrong action with a given probability. We study the ability of the system to converge towards a common working policy through the collaborative learning process based on the number of experiences from each of the agents to be aggregated for each policy update, together with the fraction of wrong actions from agents experiencing malfunctions. Our experiments are carried out in a simulation environment using the Atari testbed for the discrete action spaces, and advantage actor-critic (A2C) for the distributed multi-agent training.<br /></p>
dc.language.isoen
dc.relation.ispartofseriesProcedia Computer Science
dc.titleUbiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces
dc.identifier.urnURN:NBN:fi-fe2021042827257
dc.relation.volume177
dc.contributor.organizationfi=PÄÄT Sulautettu elektroniikka|en=PÄÄT Embedded Electronics|
dc.contributor.organization-code2606802
dc.converis.publication-id51395531
dc.converis.urlhttps://research.utu.fi/converis/portal/Publication/51395531
dc.format.pagerange324
dc.format.pagerange329
dc.identifier.jour-issn1877-0509
dc.okm.affiliatedauthorLi, Qingqing
dc.okm.affiliatedauthorPeña Queralta, Jorge
dc.okm.affiliatedauthorWesterlund, Tomi
dc.okm.discipline113 Computer and information sciencesen_GB
dc.okm.discipline213 Electronic, automation and communications engineering, electronicsen_GB
dc.okm.discipline113 Tietojenkäsittely ja informaatiotieteetfi_FI
dc.okm.discipline213 Sähkö-, automaatio- ja tietoliikennetekniikka, elektroniikkafi_FI
dc.okm.internationalcopublicationnot an international co-publication
dc.okm.internationalityInternational publication
dc.okm.typeConference proceedings article
dc.publisher.countryNetherlandsen_GB
dc.publisher.countryAlankomaatfi_FI
dc.publisher.country-codeNL
dc.relation.conferenceInternational Conference on Emerging Ubiquitous Systems and Pervasive Networks
dc.relation.doi10.1016/j.procs.2020.10.043
dc.relation.ispartofjournalProcedia Computer Science
dc.title.bookThe 11th International Conference on Emerging Ubiquitous Systems and Pervasive Networks (EUSPN 2020) / The 10th International Conference on Current and Future Trends of Information and Communication Technologies in Healthcare (ICTH 2020) / Affiliated Works
dc.year.issued2020


Aineistoon kuuluvat tiedostot

Thumbnail

Aineisto kuuluu seuraaviin kokoelmiin

Näytä suppeat kuvailutiedot