У меня есть проект библиотеки классов .net framework (v4.8), который содержит 18 модульных тестов xUnit.net, предназначенных для другого проект.
На моем сервере Azure DevOps (ADOS) pipeline У меня довольно стандартная задача Visual Studio Test, которая находит тестовую сборку с модулем тестирует и запускает их.
Когда я включаю функцию «Запускать только затронутые тесты» в задаче конвейера, ADOS запускает их все в первой сборке, что является нормальным явлением, поскольку это становится базовой сборкой.
На второй сборке конвейера все становится странно. ADOS показывает, что сборка содержит 36 модульных тестов, из которых 18 «не затронуты» (и пропущены = нормальные), а 18 выполнены и «пройдены».
У меня вопрос: Почему мои модульные тесты выполняются, если они признаны "не затронутыми"? Как я могу это исправить?
PS - Прилагаю ниже скриншот и журнал задач и заметил две интересные вещи:
- Похоже, что этап отбора тестов в самом начале происходил дважды.
- # P7 #
# P8 #
# P9 #
Скриншот:
Журнал задач Visual Studio Test:
2019-07-31T13:25:33.3027955Z ##[section]Starting: Redacted tests
2019-07-31T13:25:33.3042189Z ==============================================================================
2019-07-31T13:25:33.3042843Z Task : Visual Studio Test
2019-07-31T13:25:33.3043811Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2019-07-31T13:25:33.3044138Z Version : 2.143.8
2019-07-31T13:25:33.3044323Z Author : Microsoft Corporation
2019-07-31T13:25:33.3044558Z Help : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2019-07-31T13:25:33.3044783Z ==============================================================================
2019-07-31T13:25:35.5490506Z SystemVssConnection exists true
2019-07-31T13:25:35.5492434Z SystemVssConnection exists true
2019-07-31T13:25:35.6518950Z Running tests using vstest.console.exe runner.
2019-07-31T13:25:35.6519796Z ======================================================
2019-07-31T13:25:35.6520669Z Test selector : Test assemblies
2019-07-31T13:25:35.6521225Z Test filter criteria : null
2019-07-31T13:25:35.6521656Z Search folder : C:\_work\169\s
2019-07-31T13:25:35.6523219Z VisualStudio version selected for test execution : latest
2019-07-31T13:25:35.6888410Z Run in parallel : false
2019-07-31T13:25:35.6933473Z Run in isolation : false
2019-07-31T13:25:35.6934221Z Path to custom adapters : null
2019-07-31T13:25:35.6953986Z Other console options : null
2019-07-31T13:25:35.6954775Z Code coverage enabled : false
2019-07-31T13:25:35.6958141Z Diagnostics enabled : false
2019-07-31T13:25:35.7006840Z SystemVssConnection exists true
2019-07-31T13:25:35.7068611Z Run the tests locally using vstest.console.exe
2019-07-31T13:25:35.7069443Z ========================================================
2019-07-31T13:25:35.7090912Z Test selector : Test assemblies
2019-07-31T13:25:35.7101985Z Test assemblies : **\*test*.dll,!**\*TestAdapter.dll,!**\obj\**
2019-07-31T13:25:35.7102753Z Test filter criteria : null
2019-07-31T13:25:35.7114572Z Search folder : C:\_work\169\s
2019-07-31T13:25:35.7121444Z Run settings file : C:\_work\169\s
2019-07-31T13:25:35.7129106Z Run in parallel : false
2019-07-31T13:25:35.7135940Z Run in isolation : false
2019-07-31T13:25:35.7185224Z Path to custom adapters : null
2019-07-31T13:25:35.7186782Z Other console options : null
2019-07-31T13:25:35.7193511Z Code coverage enabled : false
2019-07-31T13:25:35.7195285Z Diagnostics enabled : false
2019-07-31T13:25:35.7202436Z Rerun failed tests: false
2019-07-31T13:25:35.7213303Z VisualStudio version selected for test execution : latest
2019-07-31T13:25:35.9796440Z ========================================================
2019-07-31T13:25:48.9123895Z ======================================================
2019-07-31T13:25:49.1259524Z [command]C:\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.143.8\TestSelector\TestSelector.exe PublishCodeChanges
2019-07-31T13:25:51.5460715Z Is pull request flow: False
2019-07-31T13:25:51.5461897Z
2019-07-31T13:25:51.5462363Z ***************************** Section 'Get Files that were changed' Starting *****************************
2019-07-31T13:25:51.5462753Z Baseline build: http://redacted/tfs/Redacted/390eb9aa-e420-4922-8ef4-bb12632dda52/_build/results?buildId=26442
2019-07-31T13:25:51.5463129Z Baseline build id = 26442
2019-07-31T13:25:51.5463438Z Total files changed = 0
2019-07-31T13:25:51.6444164Z ***************************** Section 'Get Files that were changed' Ended ********************************
2019-07-31T13:25:51.6445042Z
2019-07-31T13:26:04.3435720Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" C:\_work\169\s\Redacted\Redacted.Tests\bin\Release\Redacted.Tests.dll /ListFullyQualifiedTests /ListTestsTargetPath:C:\_work\_temp\b76cadd0-b396-11e9-84b4-7144b5b4b30f.txt "/TestAdapterPath:\"C:\_work\169\s\""
2019-07-31T13:26:07.4913814Z Microsoft (R) Test Execution Command Line Tool Version 15.9.0
2019-07-31T13:26:07.4915501Z Copyright (c) Microsoft Corporation. All rights reserved.
2019-07-31T13:26:07.4916535Z
2019-07-31T13:26:07.4917502Z [xUnit.net 00:00:00.5867523] Discovering: Redacted.Tests
2019-07-31T13:26:07.4918503Z [xUnit.net 00:00:00.5867523] Discovering: Redacted.Tests
2019-07-31T13:26:07.4919413Z [xUnit.net 00:00:00.7084444] Discovered: Redacted.Tests
2019-07-31T13:26:07.4920310Z [xUnit.net 00:00:00.7084444] Discovered: Redacted.Tests
2019-07-31T13:26:07.4921380Z Microsoft (R) Test Execution Command Line Tool Version 15.9.0
2019-07-31T13:26:07.4922173Z Copyright (c) Microsoft Corporation. All rights reserved.
2019-07-31T13:26:07.4923076Z
2019-07-31T13:26:07.4924003Z [xUnit.net 00:00:00.5867523] Discovering: Redacted.Tests
2019-07-31T13:26:07.4924894Z [xUnit.net 00:00:00.5867523] Discovering: Redacted.Tests
2019-07-31T13:26:07.4925884Z [xUnit.net 00:00:00.7084444] Discovered: Redacted.Tests
2019-07-31T13:26:07.4926792Z [xUnit.net 00:00:00.7084444] Discovered: Redacted.Tests
2019-07-31T13:26:07.4927362Z
2019-07-31T13:26:07.4973829Z [command]C:\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.143.8\TestSelector\TestSelector.exe GetImpactedtests
2019-07-31T13:26:09.4002997Z
2019-07-31T13:26:09.4004545Z ***************************** Section 'Get Impacted Tests' Starting *****************************
2019-07-31T13:26:09.4005365Z Following tests are impacted.
2019-07-31T13:26:09.4005940Z Redacted.Redacted.Tests.RedactedTests.Test1
2019-07-31T13:26:09.4006587Z Redacted.Redacted.Tests.RedactedTests.Test2
2019-07-31T13:26:09.4007287Z Redacted.Redacted.Tests.RedactedTests.Test3
2019-07-31T13:26:09.4007967Z Redacted.Redacted.Tests.RedactedTests.Test4
2019-07-31T13:26:09.4008425Z Redacted.Redacted.Tests.RedactedTests.Test5
2019-07-31T13:26:09.4008946Z Redacted.Redacted.Tests.RedactedTests.Test6
2019-07-31T13:26:09.4009492Z Redacted.Redacted.Tests.RedactedTests.Test7
2019-07-31T13:26:09.4010065Z Redacted.Redacted.Tests.RedactedTests.Test8
2019-07-31T13:26:09.4010533Z Redacted.Redacted.Tests.RedactedTests.Test9
2019-07-31T13:26:09.4011052Z Redacted.Redacted.Tests.RedactedTests.Test10
2019-07-31T13:26:09.4011524Z Redacted.Redacted.Tests.RedactedTests.Test11
2019-07-31T13:26:09.4013893Z Redacted.Redacted.Tests.RedactedTests.Test12
2019-07-31T13:26:09.4015028Z Redacted.Redacted.Tests.RedactedTests.Test13
2019-07-31T13:26:09.4015617Z Redacted.Redacted.Tests.RedactedTests.Test14
2019-07-31T13:26:09.4016093Z Redacted.Redacted.Tests.RedactedTests.Test15
2019-07-31T13:26:09.4016626Z Redacted.Redacted.Tests.RedactedTests.Test16
2019-07-31T13:26:09.4017177Z Redacted.Redacted.Tests.RedactedTests.Test17
2019-07-31T13:26:09.4017718Z Redacted.Redacted.Tests.RedactedTests.Test18
2019-07-31T13:26:09.5412388Z ***************************** Section 'Get Impacted Tests' Ended ********************************
2019-07-31T13:26:09.5413181Z
2019-07-31T13:26:36.7905770Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @C:\_work\_temp\adfd8492-b396-11e9-84b4-7144b5b4b30f.txt
2019-07-31T13:26:37.3869927Z Microsoft (R) Test Execution Command Line Tool Version 15.9.0
2019-07-31T13:26:37.3870828Z Copyright (c) Microsoft Corporation. All rights reserved.
2019-07-31T13:26:37.3871108Z
2019-07-31T13:26:37.3872406Z vstest.console.exe /Tests:"Redacted.Redacted.Tests.RedactedTests.Test1,Redacted.Redacted.Tests.RedactedTests.Test2,Redacted.Redacted.Tests.RedactedTests.Test3,Redacted.Redacted.Tests.RedactedTests.Test4,Redacted.Redacted.Tests.RedactedTests.Test5,Redacted.Redacted.Tests.RedactedTests.Test6,Redacted.Redacted.Tests.RedactedTests.Test7,Redacted.Redacted.Tests.RedactedTests.Test8,Redacted.Redacted.Tests.RedactedTests.Test9,Redacted.Redacted.Tests.RedactedTests.Test10,Redacted.Redacted.Tests.RedactedTests.Test11,Redacted.Redacted.Tests.RedactedTests.Test12,Redacted.Redacted.Tests.RedactedTests.Test13,Redacted.Redacted.Tests.RedactedTests.Test14,Redacted.Redacted.Tests.RedactedTests.Test15,Redacted.Redacted.Tests.RedactedTests.Test16,Redacted.Redacted.Tests.RedactedTests.Test17,Redacted.Redacted.Tests.RedactedTests.Test18"
2019-07-31T13:26:37.3874590Z "C:\_work\169\s\Redacted\Redacted.Tests\bin\Release\Redacted.Tests.dll"
2019-07-31T13:26:37.3874969Z /Settings:"C:\_work\_temp\b5e34eb0-b396-11e9-84b4-7144b5b4b30f.runsettings"
2019-07-31T13:26:37.3875446Z /logger:"trx"
2019-07-31T13:26:37.3875824Z /TestAdapterPath:"C:\_work\169\s"
2019-07-31T13:26:37.6710107Z Starting test discovery, please wait...
2019-07-31T13:26:39.6038466Z [xUnit.net 00:00:00.4368142] Discovering: Redacted.Tests
2019-07-31T13:26:39.6040363Z [xUnit.net 00:00:00.4368142] Discovering: Redacted.Tests
2019-07-31T13:26:39.7546522Z [xUnit.net 00:00:00.5937753] Discovered: Redacted.Tests
2019-07-31T13:26:39.7547684Z [xUnit.net 00:00:00.5937753] Discovered: Redacted.Tests
2019-07-31T13:26:43.7883423Z 3.1075
2019-07-31T13:26:50.1977477Z [xUnit.net 00:00:02.0480239] Starting: Redacted.Tests
2019-07-31T13:26:53.3389390Z 718.1859
2019-07-31T13:26:53.5472252Z Passed Test1
2019-07-31T13:26:53.5603376Z 56.5764
2019-07-31T13:26:53.6326306Z 54.9699
2019-07-31T13:26:53.7154878Z 64.2715
2019-07-31T13:26:53.8069205Z 73.892
2019-07-31T13:26:53.9014648Z 68.4937
2019-07-31T13:26:54.0023435Z 80.3846
2019-07-31T13:26:54.1314930Z 100.5241
2019-07-31T13:26:54.2364680Z 80.3842
2019-07-31T13:26:54.3291170Z 69.7939
2019-07-31T13:26:54.3428291Z Passed Test2
2019-07-31T13:26:54.3429229Z Passed Test3
2019-07-31T13:26:54.3429629Z Passed Test4
2019-07-31T13:26:54.3431016Z Passed Test5
2019-07-31T13:26:54.3431880Z Passed Test6
2019-07-31T13:26:54.3433438Z Passed Test7
2019-07-31T13:26:54.3434523Z Passed Test8
2019-07-31T13:26:54.3435127Z Passed Test9
2019-07-31T13:26:54.3435892Z Passed Test10
2019-07-31T13:26:54.4130603Z 59.9983
2019-07-31T13:26:54.4911591Z 60.3922
2019-07-31T13:26:54.5695728Z 54.5636
2019-07-31T13:26:54.6410513Z 55.6404
2019-07-31T13:26:54.7216412Z 62.6562
2019-07-31T13:26:54.7934494Z 53.8405
2019-07-31T13:26:54.9052845Z 92.4184
2019-07-31T13:26:55.0215033Z 90.1679
2019-07-31T13:26:55.0358313Z [xUnit.net 00:00:06.9133992] Finished: Redacted.Tests
2019-07-31T13:26:55.6290009Z Passed Test11
2019-07-31T13:26:55.6291164Z Passed Test12
2019-07-31T13:26:55.6292127Z Passed Test13
2019-07-31T13:26:55.6292638Z Passed Test14
2019-07-31T13:26:55.6292965Z Passed Test15
2019-07-31T13:26:55.6293222Z Passed Test16
2019-07-31T13:26:55.6293491Z Passed Test17
2019-07-31T13:26:55.6293753Z Passed Test18
2019-07-31T13:26:55.7055753Z Results File: C:\_work\169\s\TestResults\mossadmin_SRVSP2013D23_2019-07-31_16_26_53.trx
2019-07-31T13:26:55.7098705Z
2019-07-31T13:26:55.7103676Z Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
2019-07-31T13:26:55.7105019Z Test Run Successful.
2019-07-31T13:26:55.7126127Z Test execution time: 11.7080 Seconds
2019-07-31T13:26:55.8276786Z [command]C:\_work\_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.143.8\TestSelector\TestSelector.exe UpdateTestResults
2019-07-31T13:26:59.0886259Z
2019-07-31T13:26:59.0887559Z ***************************** Section 'Update test results' Starting *****************************
2019-07-31T13:26:59.0888142Z Test results remaining to be updated: 18
2019-07-31T13:26:59.0888434Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test1 to test run.
2019-07-31T13:26:59.0889274Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test2 to test run.
2019-07-31T13:26:59.0890060Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test3 to test run.
2019-07-31T13:26:59.0890582Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test4 to test run.
2019-07-31T13:26:59.0891197Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test5 to test run.
2019-07-31T13:26:59.0891663Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test6 to test run.
2019-07-31T13:26:59.0894448Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test7 to test run.
2019-07-31T13:26:59.0895370Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test8 to test run.
2019-07-31T13:26:59.0895890Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test9 to test run.
2019-07-31T13:26:59.0896359Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test10 to test run.
2019-07-31T13:26:59.0896879Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test11 to test run.
2019-07-31T13:26:59.0897955Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test12 to test run.
2019-07-31T13:26:59.0898632Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test13 to test run.
2019-07-31T13:26:59.0899585Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test14 to test run.
2019-07-31T13:26:59.0900632Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test15 to test run.
2019-07-31T13:26:59.0901513Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test16 to test run.
2019-07-31T13:26:59.0902139Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test17 to test run.
2019-07-31T13:26:59.0902833Z TcmTestResultPublisher.ConvertResults : No matching tcm test case result found for test case result given by agent to publisher. Adding test case : Redacted.Redacted.Tests.RedactedTests.Test18 to test run.
2019-07-31T13:26:59.0903437Z UnMatched Results found. Adding them to test run.
2019-07-31T13:26:59.0904228Z Publishing test results to test run '2490'.
2019-07-31T13:26:59.0905286Z TestResults To Publish 18, Test run id:2490
2019-07-31T13:26:59.0905702Z Test results publishing 18, remaining: 0. Test run id: 2490
2019-07-31T13:26:59.0906262Z Publishing test results to test run '2490'.
2019-07-31T13:26:59.0906568Z TestResults To Publish 18, Test run id:2490
2019-07-31T13:26:59.0907212Z Test results publishing 18, remaining: 0. Test run id: 2490
2019-07-31T13:26:59.0907563Z Total not impacted test: 18
2019-07-31T13:26:59.1801393Z ***************************** Section 'Update test results' Ended ********************************
2019-07-31T13:26:59.1802118Z
2019-07-31T13:26:59.2962424Z ##[section]Finishing: Redacted tests
