delay is not respected with runComposeUiTest

[igorescodro]

Describe the bug
Currently, I'm having different behaviors with UI tests in Android and iOS: the delay calls are being respected in Android but ignored in iOS. I found this issue because my app has time-based information, and I created a LaunchedEffect to re-trigger recompositions after 1 minute. This causes the test to loop with infinite recompositions until the test is manually stopped.

The code works fine in Android and iOS for the debug/release builds, but fails on iOS during tests with runComposeUiTest.

Affected platforms

• iOS (test/iosSimulatorArm64Test)

Versions

• Libraries:
  • Compose Multiplatform version: 1.6.2
• Kotlin version: 1.9.23
• OS version(s) (required for Desktop and iOS issues): macOS Sonoma 14.4.1
• OS architecture (x86 or arm64): arm64
• Device (model or simulator for iOS issues): iOS Simulator (iPhone 15)
• JDK (for desktop issues): openjdk 17.0.11 2024-04-16

To Reproduce
Steps to reproduce the behavior:

1. Run the code snippet below with :iosSimulatorArm64Test
2. The test passes
3. Run the code snippet below with :connectedAndroidTest
4. See infinite loop due to the recompositions

import androidx.compose.material.Text
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.test.ExperimentalTestApi
import androidx.compose.ui.test.runComposeUiTest
import kotlinx.coroutines.delay
import kotlinx.datetime.Clock
import kotlin.test.Test

internal class DelayTest {
    @OptIn(ExperimentalTestApi::class)
    @Test
    fun test_delay() = runComposeUiTest {
        setContent {
            println("setContent ${getCurrentTime()}")
            var refreshKey by mutableStateOf(0)
            LaunchedEffect(Unit) {
                while (true) {
                    println("LaunchedEffect - key $refreshKey - ${getCurrentTime()}")
                    delay(10_000)
                    refreshKey++
                }
            }
            Text("Hello, World!")
            assertTrue(true)
        }
    }

    private fun getCurrentTime(): Long {
        return Clock.System.now().toEpochMilliseconds()
    }
}

Expected behavior
The iOS test should have the same result as the Android counterpart, and do not skip delays in UI-related tests.

iOS output

setContent 1715548285226
LaunchedEffect - key 0 - 1715548285229
LaunchedEffect - key 1 - 1715548285267
LaunchedEffect - key 2 - 1715548285281
LaunchedEffect - key 3 - 1715548285295
LaunchedEffect - key 4 - 1715548285309
LaunchedEffect - key 5 - 1715548285324
LaunchedEffect - key 6 - 1715548285338
LaunchedEffect - key 7 - 1715548285352
LaunchedEffect - key 8 - 1715548285366
LaunchedEffect - key 9 - 1715548285381
LaunchedEffect - key 10 - 1715548285402
[...]

[eymar]

Reproduced the on Desktop as well:

• delay is not respected
• while loop never stops

It makes sense because runComposeUiTest is implemented by SkikoComposeUiTest on all platforms except Android.

My undersanding of the behaviour:
SkikoComposeUiTest uses UnconfinedTestDispatcher() and it's applied to composition too. UnconfinedTestDispatcher doesn't follow the real time, it uses virtual time instead to allow "faster" tests for time-based behaviour. By defualt it skips delays.

As I remember, it was possible to disable autoAdvance of time and to manage it manually by adding:
mainClock.autoAdvance = false at the top of your test.

I tried it now, and indeed it makes the test complete. But it seems I can't manage the time manually: mainClock.advanceTimeBy(20000) - doesn't make the coroutine in LaunchedEffect resume its execution.

Perhaps @m-sasha, Could you please take a look here?
I think even if the workaround with disabling autoAdvance was helpful, it would still mean the code for android vs non-android tests is not identical?
I remember when porting some tests we indeed had to change the code a bit, but I'm not sure about current state of the tests API and its implementation.

[m-sasha]

Everything seems to be working as expected here (tested with compose-plugin = "1.6.10").

delay is intentionally skipped in UI tests, on both Android and in Compose Multiplatform. On Android, the existence of suspended coroutines in a LaunchedEffect does not prevent waitForIdle from returning. Because of this, the test finishes on Android. I've reported this here, and it appears this has been reported by others as well.

Using mainClock.autoAdvance = false and mainClock.advanceTimeBy appears to also be working correctly. The code below prints once when run without mainClock.advanceTimeBy(11_000) and twice with.

internal class DelayTest {

    @OptIn(ExperimentalTestApi::class)
    @Test
    fun test_delay() = runComposeUiTest {
        mainClock.autoAdvance = false
        setContent {
            println("setContent ${getCurrentTime()}")
            var refreshKey by mutableStateOf(0)
            LaunchedEffect(Unit) {
                while (true) {
                    println("LaunchedEffect - key $refreshKey - ${getCurrentTime()}")
                    delay(10_000)
                    refreshKey++
                }
            }
            Text("Hello, World!")
            assertTrue(true)
        }

//        mainClock.advanceTimeBy(11_000)
    }

    private fun getCurrentTime(): Long {
        return Clock.System.now().toEpochMilliseconds()
    }
}