© Krzysztof Siegieda. Article available under the CC BY-SA 4.0 licence
Simulation studies of algorithmic collusion often summarise results through average prices and profts. Such summaries conceal the variety and distribution of distinct prices, cycles, and collusive regimes. This limitation is salient given that game theory predicts multiple equilibria, implying a distribution rather than a single outcome. This paper examines the heterogeneity hidden by aggregate metrics in a sequential pricing duopoly with Q learning agents. Using an established simulation framework for studying actions of pricing algorithms, the simulations cover three memory confgurations: symmetric one period, one period versus none, and one period versus two period memory. K means cluster analysis of steady state prices reveals a spectrum: monopoly focal price equilibria, collusive focal price equilibria at non monopoly grid points and incomplete Edgeworth cycles with supra competitive averages. These regimes are consistent with the multiplicity of the Markov perfect equilibria in the game theory literature. In a symmetric baseline, the monopoly outcome is a minority. Memory asymmetry eliminates or reduces cycling and increases the share of fxed price outcomes, stabilising rather than breaking collusion. Aggregate numbers collapse qualitatively distinct results into a single fgure; cluster analysis uncovers the hidden heterogeneity behind those averages and shows that algorithmic coordination spans a spectrum of regimes rather than a single outcome. For the competition policy, instruments that distinguish between pricing regimes are more informative than price level benchmarks alone.
algorithmic collusion, cluster analysis, reinforcement learning, Q-learning, pricing algorithms
K21, L13, L41, C63, C73
Abada, I., Harrington, J. E., Lambin, X., & Meylahn, J. M. (2025). Algorithmic Collusion: Where are we and where should we be going? (Working paper). https://dx.doi.org/10.2139/ssrn.4891033.
Abada, I., & Lambin, X. (2020). Artificial intelligence: Can seemingly collusive outcomes be avoided? (Working paper). https://dx.doi.org/10.2139/ssrn.3559308.
Abada, I., Lambin, X., & Tchakarov, N. (2024). Collusion by mistake: Does algorithmic sophistication drive supra-competitive profits?. European Journal of Operational Research, 318(3), 927–953. https://doi.org/10.1016/j.ejor.2024.06.006.
Bichler, M., Durmann, J., & Oberlechner, M. (2024). Online optimization algorithms in repeated price competition: Equilibrium learning and algorithmic collusion. https://doi.org/10.48550/arXiv.2412.15707.
den Boer, A. V., Meylahn, J. M., & Schinkel, M. P. (2022). Artificial collusion: Examining supracompetitive pricing by Q-learning algorithms (Amsterdam Law School Research Paper No. 25). https://dx.doi.org/10.2139/ssrn.4213600.
Byrne, D. P., & de Roos, N. (2019). Learning to coordinate: A study in retail gasoline. The American Economic Review, 109(2), 591–619. https://doi.org/10.1257/aer.20170116.
Calvano, E., Calzolari, G., Denicolo, V., Harrington, J. E., & Pastorello, S. (2020). Protecting consumers from collusive prices due to AI. Science, 370(6520), 1040–1042. https://doi.org/10.1126/science.abe3796.
Calvano, E., Calzolari, G., Denicolo, V., & Pastorello, S. (2020). Artificial intelligence, algorithmic pricing, and collusion. The American Economic Review, 110(10), 3267–3297. https://doi.org/10.1257/aer.20190623.
Eckert, A. (2013). Empirical studies of gasoline retailing: A guide to the literature. Journal of Economic Surveys, 27(1), 140–166. https://doi.org/10.1111/j.1467-6419.2011.00698.x.
Ezrachi, A., & Stucke, M. E. (2016). Virtual Competition. The promise and perils of the algorithmdriven economy. Harvard University Press.
Harrington, J. E. (2018). Developing competition law for collusion by autonomous artificial agents. Journal of Competition Law & Economics, 14(3), 331–363. https://doi.org/10.1093/joclec/nhy016.
James, G., Witten, D., Hastie, T., Tibshirani, R., & Taylor, J. (2023). An Introduction to Statistical Learning with Applications in Python. Springer. https://doi.org/10.1007/978-3-031-38747-0_2.
Kamiński, B., & Zawisza, M. (2012). Receptury w R. Podręcznik dla ekonomistów. Oficyna Wydawnicza SGH.
Klein, T. (2021). Autonomous algorithmic collusion: Q-learning under sequential pricing. The RAND Journal of Economics, 52(3), 538–558. https://doi.org/10.1111/1756-2171.12383.
Mas-Colell, A., Whinston, M. D., & Green, J. R. (1995). Microeconomic Theory. Oxford University Press.
MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In L. M. Le Cam & J. Neyman (Eds.), Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (Vol. 1, pp. 281–297). University of California Press.
Maskin, E., & Tirole, J. (1988). A theory of dynamic oligopoly, II: Price competition, kinked demand curves, and Edgeworth cycles. Econometrica, 56(3), 571–599. https://doi.org/10.2307/1911701.
Mehra, S. K. (2016). Antitrust and the Robo-Seller: Competition in the Time of Algorithms. Minnesota Law Review, 100, 1323–1375.
Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53–65. https://doi.org/10.1016/0377-0427(87)90125-7.
Siegieda, K., & Zawisza, M. (2026). Memory Arms Race in Q-learning: How Memory Asymmetry Shapes Algorithmic Collusion in Sequential Pricing (Working paper). https://dx.doi.org/10.2139/ssrn.6479701.
Waltman, L., & Kaymak, U. (2008). Q-learning agents in a Cournot oligopoly model. Journal of Economic Dynamics & Control, 32(10), 3275–3293. https://doi.org/10.1016/j.jedc.2008.01.003.
Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3–4), 279–292. https://doi.org/10.1007/BF00992698.
