#Midscene AndroidWorld Benchmark Report

This is Midscene's test report for the AndroidWorld benchmark. In this run, Midscene achieved Pass@1 93.10%, Pass@2 95.69%, and Pass@3 97.41%.

#Run Configuration

#Stability Improvements

The following changes did not change the task intent. They made benchmark execution more stable by reducing browser rendering races, stale accessibility reads, and setup timing issues.

#Validation Condition Updates

The following AndroidWorld validation checks were changed on the main branch used for this benchmark:

#Report Files

Detailed reports are listed below for reference.

Round 1 (115 reports · 108 PASS · 7 FAIL)

#	Task	Status	Report
1	AudioRecorderRecordAudio	PASS	report
2	AudioRecorderRecordAudioWithFileName	PASS	report
3	BrowserDraw	PASS	report
4	BrowserMaze	PASS	report
5	BrowserMultiply	PASS	report
6	CameraTakePhoto	PASS	report
7	CameraTakeVideo	PASS	report
8	ClockStopWatchPausedVerify	PASS	report
9	ClockStopWatchRunning	PASS	report
10	ClockTimerEntry	PASS	report
11	ContactsAddContact	PASS	report
12	ContactsNewContactDraft	PASS	report
13	ExpenseAddMultiple	PASS	report
14	ExpenseAddMultipleFromGallery	PASS	report
15	ExpenseAddMultipleFromMarkor	FAIL	report
16	ExpenseAddSingle	PASS	report
17	ExpenseDeleteDuplicates	PASS	report
18	ExpenseDeleteDuplicates2	PASS	report
19	ExpenseDeleteMultiple	PASS	report
20	ExpenseDeleteMultiple2	PASS	report
21	ExpenseDeleteSingle	PASS	report
22	FilesDeleteFile	PASS	report
23	FilesMoveFile	PASS	report
24	MarkorAddNoteHeader	PASS	report
25	MarkorChangeNoteContent	PASS	report
26	MarkorCreateFolder	PASS	report
27	MarkorCreateNote	PASS	report
28	MarkorCreateNoteAndSms	PASS	report
29	MarkorCreateNoteFromClipboard	PASS	report
30	MarkorDeleteAllNotes	PASS	report
31	MarkorDeleteNewestNote	PASS	report
32	MarkorDeleteNote	PASS	report
33	MarkorEditNote	PASS	report
34	MarkorMergeNotes	PASS	report
35	MarkorMoveNote	PASS	report
36	MarkorTranscribeReceipt	PASS	report
37	MarkorTranscribeVideo	FAIL	report
38	OpenAppTaskEval	PASS	report
39	OsmAndFavorite	PASS	report
40	OsmAndMarker	FAIL	report
42	RecipeAddMultipleRecipes	PASS	report
43	RecipeAddMultipleRecipesFromImage	FAIL	report
44	RecipeAddMultipleRecipesFromMarkor	PASS	report
45	RecipeAddMultipleRecipesFromMarkor2	PASS	report
46	RecipeAddSingleRecipe	PASS	report
47	RecipeDeleteDuplicateRecipes	PASS	report
48	RecipeDeleteDuplicateRecipes2	FAIL	report
49	RecipeDeleteDuplicateRecipes3	FAIL	report
50	RecipeDeleteMultipleRecipes	PASS	report
51	RecipeDeleteMultipleRecipesWithConstraint	PASS	report
52	RecipeDeleteMultipleRecipesWithNoise	PASS	report
53	RecipeDeleteSingleRecipe	PASS	report
54	RecipeDeleteSingleWithRecipeWithNoise	PASS	report
55	RetroCreatePlaylist	PASS	report
56	RetroPlayingQueue	PASS	report
57	RetroPlaylistDuration	PASS	report
58	RetroSavePlaylist	PASS	report
59	SaveCopyOfReceiptTaskEval	PASS	report
60	SimpleCalendarAddOneEvent	PASS	report
61	SimpleCalendarAddOneEventInTwoWeeks	PASS	report
62	SimpleCalendarAddOneEventRelativeDay	PASS	report
63	SimpleCalendarAddOneEventTomorrow	PASS	report
64	SimpleCalendarAddRepeatingEvent	PASS	report
65	SimpleCalendarDeleteEvents	PASS	report
66	SimpleCalendarDeleteEventsOnRelativeDay	PASS	report
67	SimpleCalendarDeleteOneEvent	PASS	report
68	SimpleDrawProCreateDrawing	PASS	report
69	SimpleSmsReply	PASS	report
70	SimpleSmsReplyMostRecent	PASS	report
71	SimpleSmsResend	PASS	report
72	SimpleSmsSend	PASS	report
73	SimpleSmsSendClipboardContent	PASS	report
74	SimpleSmsSendReceivedAddress	PASS	report
75	SystemBluetoothTurnOff	PASS	report
76	SystemBluetoothTurnOffVerify	PASS	report
77	SystemBluetoothTurnOn	PASS	report
78	SystemBluetoothTurnOnVerify	PASS	report
79	SystemBrightnessMax	PASS	report
80	SystemBrightnessMaxVerify	PASS	report
81	SystemBrightnessMin	PASS	report
82	SystemBrightnessMinVerify	PASS	report
83	SystemCopyToClipboard	FAIL	report
84	SystemWifiTurnOff	PASS	report
85	SystemWifiTurnOffVerify	PASS	report
86	SystemWifiTurnOn	PASS	report
87	SystemWifiTurnOnVerify	PASS	report
88	TurnOffWifiAndTurnOnBluetooth	PASS	report
89	TurnOnWifiAndOpenApp	PASS	report
90	VlcCreatePlaylist	PASS	report
91	VlcCreateTwoPlaylists	PASS	report
92	NotesIsTodo	PASS	report
93	NotesMeetingAttendeeCount	PASS	report
94	NotesRecipeIngredientCount	PASS	report
95	NotesTodoItemCount	PASS	report
96	SimpleCalendarAnyEventsOnDate	PASS	report
97	SimpleCalendarEventOnDateAtTime	PASS	report
98	SimpleCalendarEventsInNextWeek	PASS	report
99	SimpleCalendarEventsInTimeRange	PASS	report
100	SimpleCalendarEventsOnDate	PASS	report
101	SimpleCalendarFirstEventAfterStartTime	PASS	report
102	SimpleCalendarLocationOfEvent	PASS	report
103	SimpleCalendarNextEvent	PASS	report
104	SimpleCalendarNextMeetingWithPerson	PASS	report
105	SportsTrackerActivitiesCountForWeek	PASS	report
106	SportsTrackerActivitiesOnDate	PASS	report
107	SportsTrackerActivityDuration	PASS	report
108	SportsTrackerLongestDistanceActivity	PASS	report
109	SportsTrackerTotalDistanceForCategoryOverInterval	PASS	report
110	SportsTrackerTotalDurationForCategoryThisWeek	PASS	report
111	TasksCompletedTasksForDate	PASS	report
112	TasksDueNextWeek	PASS	report
113	TasksDueOnDate	PASS	report
114	TasksHighPriorityTasks	PASS	report
115	TasksHighPriorityTasksDueOnDate	PASS	report
116	TasksIncompleteTasksOnDate	PASS	report

Round 2 (7 reports · 3 PASS · 4 FAIL)

#	Task	Status	Report
37	MarkorTranscribeVideo	FAIL	report
40	OsmAndMarker	FAIL	report
41	OsmAndTrack	PASS	report
43	RecipeAddMultipleRecipesFromImage	PASS	report
48	RecipeDeleteDuplicateRecipes2	FAIL	report
49	RecipeDeleteDuplicateRecipes3	FAIL	report
83	SystemCopyToClipboard	PASS	report

Round 3 (5 reports · 2 PASS · 3 FAIL)

#	Task	Status	Report
15	ExpenseAddMultipleFromMarkor	PASS	report
37	MarkorTranscribeVideo	FAIL	report
40	OsmAndMarker	PASS	report
48	RecipeDeleteDuplicateRecipes2	FAIL	report
49	RecipeDeleteDuplicateRecipes3	FAIL	report