Start a new topic

Using 'Run All Tests' with multiple TA Test Modules will throw an error creating tests results html

Steps:

  1. New TA-VS Project
  2. Create Two Test Modules
  3. Create 1 simple test in each module (no ui, just set variable and maybe check)
  4. (note- I made one pass and one fail, not required to repo bug)
  5. In VisualStudio Use Test | Run | All Tests menu (DO NOT HAVE AN OPEN .MOD)

Results:
All the tests run, but only the first .mod generates a Test Results HTML in the Results Folder. The second .MOD file is not recorded in the results and a pop-up dialog is displayed (see attachment).


Additional Information:


Forgot to mention, If I select the .MOD and run the tests only for that mod everything works fine. Its only when I use the RUN ALL TESTS option from Visual Studio that this error happens.


Attached in the error that shows up in the test itself, from Visual Studio.



Hi Michael,


To run TA tests in Visual Studio, you want to click the "Run" button on the TestArchitect toolbar.


Let me know if this solves your issue.


Thanks,


Srish

This does not solve the issue. The issue is a pretty simple bug where the test results can't be written when multiple modules are run together.  The most likely cause is the first module still has a reference to a file that the next module is trying to use.


The "Run" button you are talking about is to run only the open and selected Test Module.


Visual Studio lets you run all the modules in order (imagine if you have many modules and must run them manually one at a time)


I think this is a real bug that should get in front of the dev team.

Hey Michael,

 

I'd maybe follow through on your idea about a reference being left open...the issue might be in the design of your tests and/or in LogiGears handling of what you're trying to do. I have an example project with multiple V6 based test modules, logging in and out of the product...opening reports etc....and when I run ALL, like you did, everything runs and reports just fine. My tests do not do anything with the setting of values in C# scripted actions...but, they do call a couple scripted actions to run our legacy Hybrid Coded UI tests.

 

Also, I kindof also agree with Srish a bit on his reply too. When you're using the Tests | Run All menu, you're running the straight Coded UI Test Methods...not the TA4VS Test Modules. So, you're not trully running the test modules, you're running the the underlinying coded ui "bits" of them...and maybe that doesn't properlly allow for the calling of those custom value setting calls you're making???

I had reported this issue also, before they invented the forum.  I have over 80 tests that get run twice a day.  Not being able to run them as a set, vs individually, will greatly impact what I'm trying to do.  Can't really see putting all the tests into one TA module because some of them are data-driven, some are not.

ADMIN

Hi Michael,

 

Thank you for raising question for us and other TA4VS users to discuss. 

 

If you are using Microsoft Test Manger in your project, you can run multiple test modules by collecting all tests to be run into a "test suite" in MTM. Each test case in the MTM test suite need to be associated with your test case in the TA4VS test module. The detailed steps to associate MTM and TA4VS test cases can be found at in TA4VS Help, topic "Associating test cases".

 

You can create your "Master Test Suite" on MTM in two ways:

- Create a test suite then create test cases or add test cases to the test suite. By this way you can arrange your test cases to run follow the order of test cases in TA4VS.

- Create a "query-based suite" and query all test cases that your created before. MTM supports users many ways to query. You can use "Area Path" for example: set all your tests in a test module with the same "Area Path" then query them. This way is faster but you cannot arrange your test cases to run in the order you would like to.

 

Hope it helps.

I am also checking with the product team to see if it is possible to support run multiple tests from VS without using MTM. I will keep you posted on this. 


Please let us know if you need further assistance with this questions.

ADMIN

Hi Michael, Dennis, Joe,


I have worked with the product team on this feature request. We support this feature in December release. We will keep you posted when the build is ready.


Please stay tune.


Thank you and regards,

Van

ADMIN

Hi all, 


Dennis just sent a feedback that the update 1.2.1 did have the Run All function as well as Filter. Thank you so much for that, Dennis. 


Hi Michael, Joe,


Could you please try out and let us know if it is working for you too?


Regards,

Van.


Login or Signup to post a comment