Monday, March 16, 2015

JUnit 4 Vs TestNG – Comparison

TestNG Introduction

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionality that make it more powerful and easier to use.
TestNG is an open source automated testing framework; where NG of TestNG means Next Generation. TestNG is similar to JUnit but it is much more powerful than JUnit but still it’s inspired by JUnit. It is designed to be better than JUnit, especially when testing integrated classes. Pay special thanks to Cedric Beust who is the creator of TestNG.
TestNG eliminates most of the limitations of the older framework and gives the developer the ability to write more flexible and powerful tests with help of easy annotations, grouping, sequencing & parameterizing.

Benefits of TestNG

There are number of benefits of TestNG but from Selenium perspective, major advantages of TestNG are :
1) It gives the ability to produce HTML Reports of execution
2) Annotations made testers life easy
3) Test cases can be Grouped & Prioritized more easily
4) Parallel testing is possible
5) Generates Logs
6) Data Parameteriz ation is possible

Test Case Writing

Writing a test in TestNG is quite simple and basically involves following steps:
Step 1 – Write the business logic of the test
Step 2 – Insert TestNG annotations in the code
Step 3 – Add the information about your test (e.g. the class names, methods names, groups names etc…) in a testng.xml file
Step 4 – Run TestNG

Annotations in TestNG

@BeforeSuite: The annotated method will be run before all tests in this suite have run.
@AfterSuite: The annotated method will be run after all tests in this suite have run.
@BeforeTest: The annotated method will be run before any test method belonging to the classes inside the tag is run.
@AfterTest: The annotated method will be run after all the test methods belonging to the classes inside the tag have run.
@BeforeGroups: The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.
@AfterGroups: The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.
@BeforeClass: The annotated method will be run before the first test method in the current class is invoked.
@AfterClass: The annotated method will be run after all the test methods in the current class have been run.
@BeforeMethod: The annotated method will be run before each test method.
@AfterMethod: The annotated method will be run after each test method.
@Test: The annotated method is a part of a test case.

Benefits of using annotations

1) TestNG identifies the methods it is interested in by looking up annotations. Hence method names are not restricted to any pattern or format.
2) We can pass additional parameters to annotations.
3) Annotations are strongly typed, so the compiler will flag any mistakes right away.
4) Test classes no longer need to extend anything (such as Test Case, for JUnit 3).

List of Topics

Below is the list of TestNg topics which we are going to study in the next chapters of TestNg Tutorial:
1) TestNg Installation
2) First Test Case with TestNG
3) Grouping, Prioritization & Sequencing of Test case
4) Running Test Cases as a Test Suite
5) TestNg Reporter Logs and Assertion
6) Data driven testing with TestNG Reporter
7) TestNG Listeners

TestNG Test Suite
In any project, you will end up to a place where you need to execute so many test cases on a run. Running a set of test cases together is call executing a Test Suite. Those test cases can be dependent to each other or may have to be executed in a certain order. TestNg gives us the capability to manage our test execution.
In TestNG framework, we need to create testng.xml file to create and handle multiple test classes. This is the xml file where you will configure your test run, set test dependency, include or exclude any test, method, class or package and set priority etc.
How to do it…
Step 1 : Create a TestNG XML
1) Right click on Project folder, go to New and select ‘File‘ as shown in below image.
2) In New file wizard, add file name = ‘testng.xml‘ as shown in below given image and click on Finish button.

3) It will add testng.xml file under your project folder.

Step 2 : Write xml code ?
1) Now add below given code in your testng.xml file.
Note: You can choose any name for your Test Suite & Test Name as per your need.
2) After giving appropriate names, now your testng.xml file will looks like this:

Hope you have understood the xml code, as it is quite simple hierarchy:
Very first tag is the Suite tag<suite>, under that it is the Test tag<test> and then the Class tag<classes>. You can give any name to the suite and the test but you need to provide the correct name to the <classes> tag which is a combination of your Package name and Test Case name.

Step 3 : Execute a testng.xml
Now it’s time to run the xml. Run the test by right click on the testng.xml file and select Run As > TestNG Suite.
It will take few seconds to start the testng execution engine and soon you will notice that your test will run and complete. Once the execution is complete, you can view test execution result under the TestNg console.
This is the one simple example of creating and running testng.xml file in eclipse.

Building a Test Suite

Now when you have learned how to build the xml, now it’s time to learn how to build a Test Suite using testng.xml. It is again not a complex task, all you need to do is to add your test cases to your xml file in <classes> tag.

The above test will execute only those tests, which are mentioned in the testng.xml. The rest of the test cases under ‘automationFramework’ package will remain untouched.

TestNG Annotations

In the TestNG Introduction chapter we have came across different annotations used in TestNG Framework but so far we have used just three(Before, After & Test). All though these are the most frequently used annotations but who know how far you will go with your framework and may like to use other useful TestNG annotations.
Before that I would like you to give a small idea on Annotations hierarchy or Annotations levels in TestNG.
It says that @Test is the smallest annotation here. @Method will be executed first, before and after the execution of @Test. The same way @Class will be executed first, before and after the execution of @Method and so on.
Now with the below example it will be clear to you easily.

Output of the above code will be like this:

It is clearly visible that the @Suite annotation is the very first and the very lastly executed. Then @Test followed by @Class. Now if you notice, the @Method has executed twice. As @Test is a method in the class, hence @Method will always executed for each @Test method.

Test Case Grouping

‘Groups‘ is one more annotation of TestNG which can be used in the execution of multiple tests. Let’s say you have hundred tests of class vehicle and in it ten method of car, ten method of scooter and so on. You probably like to run all the scooter tests together in a batch. And you want all to be in a single test suite. With the help of grouping you can easily overcome this situation.
How to do it…
1) Create two methods for Car, two methods for Scooter and one method in conjunction with Car & Sedan Car.
2) Group them separately with using  (groups = { ” Group Name” })

3) Create a testng xml like this:

4) Run the test by right click on the testng.xml file and select Run As > TestNG Suite. Output will be like this in TestNg console:

 Note: We have just call the group ‘Car’ from the xml and it also executed the test for Sedan Car, as we have mentioned the ‘Car’ as well while declaring the group of Sedan Car.
Clubbing of groups is also possible, take a look at the below xml:
You can see that we have created a new Group with the name ‘All’ and include other groups in it. Then simply called the newly created group for execution. The output will be like this:


Dependent Test

Sometimes, you may need to invoke methods in a Test case in a particular order or you want to share some data and state between methods. This kind of dependency is supported by TestNG as it supports the declaration of explicit dependencies between test methods.
TestNG allows you to specify dependencies either with:
  • Using attributes dependsOnMethods in @Test annotations OR
  • Using attributes dependsOnGroups in @Test annotations.
Take a look over the below example:
The output will be like this:

TestNG-Dependent1

Multiple Tests

There will be situations when you want to put number of tests under a single test class and like to run all in single shot.  With the help of TestNG ‘@Test‘ annotations we can execute multiple tests in single TestNG file.
Take an example of four different tests under one testng class and print the test sequence on the console.
How to do it…
1) Press Ctrl+N , select “TestNG Class” under TestNG category and click Next.
Or
Right click on Test Case folder, go to TestNG and select “TestNG Class“.

2) If your project is set up and you have selected the Test Case folder before creating TestNG class then the source folder and the package name will be pre-populated on the form. Set class name as ‘TestNG‘.
Leave rest of the settings untouched, do not check for “@BeforeMethod”, “@AfterMethod” for now and click Finish. That’s it.

3) By default a new class will have only one @Test method. Add two more methods by yourself and put your code accordingly in methods. Code will look like:
This will enable you to execute all four tests with just one testng class. Take a look on the output.

Attention: By default, methods annotated by @Test are executed alphabetically. Take a look over the next topic to see how to prioritize @Test.

Sequencing & Prioritizing

You need to use the ‘priority‘ parameter, if you want the methods to be executed in your order. Parameters are keywords that modify the annotation’s function.
Let’s take the same above example and execute all @Test methods in right order. Simply assign priority to all @Test methods starting from 0(Zero).
Note: TestNG will execute the @Test annotation with the lowest priority value up to the largest.
Output of the above:

Skipping a Test Case

Think of a situation where you are required to skip one or more @Test from your testng class. In testng, you can easily able to handle this situation by setting the ‘enabled’ parameter to ‘false’ for e.g.:
@Test(enabled = false)
To use two or more parameters in a single annotation, separate them with a comma:
@Test(priority = 3, enabled = false)
Again take the same example and set the value false for the third test.
Output of the above example:

TestNG Reporters & Asserts

TestNG Reporters

TestNG is a Framework and so far we have already seen the many different powerful features of TestNG. It almost gives you all the important things you are required to complete the Framework.

TestNG Reporter Logs

TestNG also gives us the logging facility for the test. For example during the running of test case user wants some information to be logged in the console. Information could be any detail depends upon the purpose. Keeping this in mind that we are using Selenium for testing, we need the information which helps the User to understand the test steps or any failure during the test case execution. With the help of TestNG Logs it is possible to enable logging during the Selenium test case execution.
In selenium there are two types of logging. High level logging and Low level logging. In low level logging you try to produce logs for the every step you take or every action you make in your automation script. In high  level logging you just try to capture main events of your test.
Everybody has their own style of logging and I have mine too. I am also a big fan of Log4j logging and that’s why I do not mix log4j logging with testng logging but on the same side I make to use of both of its. I perform low level logging with log4j and high level logging with testng reporter logs.
How to do it…
1) Write a test case for Sign In application and implement Log4j logging on every step.
2) Insert Reporter logs on the main events of the test.
3) Run the test by right click on the test case script and select Run As > TestNG Test.
Your Log4j logging output will look like this:

But your Reporters log will look like this:

Log4j logging will help you to report a bug or steps taken during the test, on the other hand reporters log will help you to share the test status with leadership. As leadership is just interested in the test results, not the test steps.
I also use reporter’s logs on the verification during the test. For example

TestNG Asserts

TestNG also gives us the power to take decisions in the middle of the test run with the help of Asserts. With this we can put various checkpoints in the test. Asserts are the most popular and frequently used methods while creating Selenium Scripts. In selenium there will be many situations in the test where you just like to check the presence of an element. All you need to do is to put an assert statement on to it to verify its existence.
Different Asserts Statements
1) Assert.assertTrue() & Assert.assertFalse() 
Note: Assert true statement fails the test and stop the execution of the test, if the actual output is false.  Assert.assertFalse() works opposite of Assert.assertTrue(). It means that if you want your test to continue only if when some certain element is not present on the page. You will use Assert false, so it will fail the test in case of the element present on the page.
2) Assert.assertEquals()
It also works the same way like assert true and assert fail. It will also stop the execution, if the value is not equal and carry on the execution, if the value is equal.

TestNG Parameters & Data Provider

TestNG Parameters

Everybody knows the importance of Parameterization in testing and in automation testing. It allows us to automatically run a test case multiple times with different input and validation values. As Selenium Webdriver is more an automated testing framework than a ready-to-use tool, you will have to put in some effort to support data driven testing in your automated tests. I usually prefer to use Microsoft Excel as the format for storing my parameters but so many of my followers have requested to write an article on TestNG Data Provider.
TestNG again gives us another interesting feature called TestNG Parameters. TestNG lets you pass parameters directly to your test methods with your testng.xml.
How to do it…
Let me take a very simple example of LogIn application, where the username and password is required to clear the authentication.
1) Create a test on my demo OnlineStore application to perform LogIn which takes the two string argument as username & password.
2) Provide Username & Password as parameter using TestNG Annotation.
3) The parameter would be passed values from testng.xml which we will see in the next step.
Now, run the testng.xml, which will run the parameterTest method. TestNG will try to find a parameter named sUsername & sPassword.

TestNG DataProviders

When you need to pass complex parameters or parameters that need to be created from Java (complex objects, objects read from a property file or a database, etc…), in such cases parameters can be passed using Dataproviders. A Data Provider is a method annotated with @DataProvider. A Data Provider returns an array of objects.
Let us check out the same Sign In examples using Dataproviders.
How to do it…
1)  Define the method credentials() which is defined as a Dataprovider using the annotation. This method returns array of object array.
2) Add a method test() to your DataProviderTest class. This method takes two strings as input parameters.
3) Add the annotation @Test(dataProvider = “Authentication”) to this method. The attribute dataProvider is mapped to “Authentication”.
Test will look like this:
Run the test by right click on the test case script and select Run As > TestNG Test. Give it few minutes to complete the execution, once it is finished the results will look like this in the TestNg Result window.

Note: As the test data is provided two times, the above test executed two times completely.

TestNG DataProvider with Excel

Data Driven Testing

A key benefit of automating functional testing is the ability to test large volumes of data on the system quickly. But you must be able to manipulate the data sets, perform calculations, and quickly create hundreds of test iterations and permutations with minimal effort. Test Automation Frameworks must have capability to integrate with spreadsheets and provide powerful calculation features.

Apache POI (Excel)

Most commercial automated software tools on the market support some sort of data driven testing, which allows you to automatically run a test case multiple times with different input and validation values. As Selenium Webdriver is more an automated testing framework than a ready-to-use tool, you will have to put in some effort to support data driven testing in your automated tests. I usually prefer to use Microsoft Excel as the format for storing my parameters. An additional advantage of using Excel is that you can easily outsource the test data administration to someone other than yourself, someone who might have better knowledge of the test cases that need to be run and the parameters required to execute them.

TestNG Data Providers

When you need to pass complex parameters or parameters that need to be created from Java (complex objects, objects read from a property file or a database, etc…), in such cases parameters can be passed using Dataproviders. A Data Provider is a method annotated with @DataProvider. A Data Provider returns an array of objects.
Let us check out the same Sign In examples using Data Providers with Excel data sheet.

How to do it…

Here we will follow a simple step by step process to Implement Excel with TestNg Data Provider.
Step 1: Create a test case of Login Application with TestNG Data Provider.
Step 2:  Create a Test Data sheet.
Step 3: Create functions to Open & Read data from Excel
Step 4: Create a TestNg test case for accepting data from Excel using Data Provider.
Step 5: Run the test against the Test Case name in the Test Data file.

Step 1: Create a test case of LogIn Application with TestNG Data Provider

1) Create a TestNG class ‘DataProviderTest’ by Pressing Ctrl+N , select ‘Create TestNG Class’ under TestNG category and Under Annotations, check ‘DataProvider’ and click Finish.
2) By default, the DataProvider name is ‘dp’, change it to ‘Authentication’. This method returns array of object array.
3) Add a method Registration_data() to your Test class. This method takes two strings as input parameters.
4) Write script for LogIn Application under method @Test.
Test Case will look like this:

Step 2:  Create a Test Data sheet

1) Create a ‘New Package‘ file and name it as ‘testData’, by right click on the Project and select New > Package. I always place my Test Data file under separate test data folder.

2) Place a Excel file in the above created package location and save it as TestData.xlsx. Fill the data in the excel like below image:


Step 3: Create functions to Open & Read data from Excel

We need a way to open this Excel sheet and read data from it within our Selenium test script. For this purpose, I use the Apache POI library, which allows you to read, create and edit Microsoft Office-documents using Java. The classes and methods we are going to use to read data from Excel sheet are located in the org.apache.poi.hssf.usermodel package.
To see step by step process to set up Apache POI Excel, please visit Data Driven Framework.

Step 4: Create a TestNg test case for accepting data from Excel using Data Provider

1) Create a TestNG class ‘DataProviderWithExcel’ by Pressing Ctrl+N , select ‘Create TestNG Class‘ under TestNG category and Under Annotations, check ‘@BeforeMethod‘, ‘@AfterMethod‘ & ‘DataProvider‘ and click Finish.
3) Add a method Registration_data() to your Test class. This method takes two strings as input parameters.
4) Now divide the test case in to three parts :
@BeforeMethod : Launch Firefox and direct it to the Base URL
@Test : Enter Username & Password to Login, Print console message and Log out
@AfterMethod : Close Firefox browser
Test Case will look like this:

Note: This LogIn test will execute two times as there are two users credentials in data provider Array.

Step 5: Run the test against the Test Case name in the Test Data file

1) This means that your test should be run once only with the data which is mentioned against the Test Case name. To do this we need to tweak the Excel utility class, plus need to add few more function to fetch out the current Test Case name & the row number which contain the Test Case name.

Final Test Case
1) Get the Test Case name.
2) With the Test Case name, get the row number of the Excel sheet for the test.
3) Get the data from the excel sheet for the fetched test row.


Note: This will just execute your test once against the current est case data.

Multi Browser, Cross Browser & Parallel Testing using TestNG


When the time comes to turn your site from mock-up to something fully functional, you’ll want to make sure that it works great for everyone visiting your site whether they’re using Internet Explorer, Firefox, or any other browser. Testing your website with multiple combinations of browsers is known as Cross Browser testing.
Your site will look different in different browsers. That’s because browsers understand some code slightly differently. Your designer should be testing to make sure that your site works well in all modern browsers. But as a tester we need to make sure that functionality should at least tested on Internet Explorer, Firefox, Safari & Google Chrome browser.

Multi Browser Testing using Selenium TestNG

In every project it is required to perform multi-browser testing to make sure that the functionality is working as expected with every browser to give equal user experience to all of the wide range of audience. It takes a considerable time to test everything on every browser and when we have used automation to reduce the testing efforts then why don’t we perform the multi-browser testing using automation. TestNG gives us functionality to perform same test on different browsers in a simple and easy way.
How to do it…
1)Create your Script to test a LogIn application using TestNG class.
2) Pass ‘Browser Type’ as parameters using TestNG annotations to the before method of the TestNG class. This method will launch only the browser, which will be provided as parameter.

3) Create a TestNG XML for running your test. Configure the TestNG XML for passing parameters i.e. to tell which browser should be used for Running the Test.
 Note: You can set any number of Browsers here and just for the example purpose I have set up only two main browsers.
4) Now it’s time to run the xml. Run the test by right click on the testng.xml file and select Run As > TestNG Suite.

Note: TestNg will execute the test one by one. You may like to perform parallel tests, next topic will cover that.

Parallel Tests using TestNG

Using the feature provided by TestNG for Parallel Executions. just take the above example for Sign In application with two different browsers. This time all we want is to execute test in both browsers simultaneously.
Now just set the ‘parallel‘ attribute to ‘tests‘ in the above used xml and give a run again. This time you will notice that your both browsers will open almost simultaneously and your test will run in parallel.


Note: You may see some intermittent issues using parallel testing. I will not recommend you this rather run one by one only.

   

It is quite an important and the last chapter of my tutorial on TestNG. It took me a long time to figure out the perfect simple example for my viewers, as I am from a non-technical background and I hooped around so many sites to get the perfect example of listener but I was unable to find one. Everybody has pasted almost same example all over the internet. Enough talking, let’s start now.
In very non-technical term, TestNG manages everything through Suite, Test and Methods and the Listeners gives us the ability to act before and after of every Suite, Test and Methods.

Types of Listeners

There are many types of listeners available in TestNG for example IAnnotationTransformer, IAnnotationTransformer2, IConfigurable, IConfigurationListener, IConfigurationListener2, IExecutionListener, IHookable, IInvokedMethodListener, IInvokedMethodListener2, IMethodInterceptor, IReporter, ISuiteListener, ITestListener.
As far as testing concern only few can be used effectively such as :
ISuiteListener: It has two method in it onStart() & onFinish(). Whenever a class implements this listener, TestNG guarantees the end-user that it will invoke the methods onStart() and onFinish() before and after running a TestNG Suite. So before TestNG picks up your suite for execution, it first makes a call to onStart() method and runs whatever has been scripted in this method. In a similar way, it again makes a call to onFinish() method after a suite has been run.
ITestListener: The working of this listener is also exactly the same as ISuiteListerner but the only difference is that it makes the call before and after the Test not the Suite. It has seven methods in it.
onFinish(): Invoked after all the tests have run and all their Configuration methods have been called.
onStart(): Invoked after the test class is instantiated and before any configuration method is called.
onTestFailedButWithinSuccessPercentage(ITestResult result): Invoked each time a method fails but has been annotated with successPercentage and this failure still keeps it within the success percentage requested.
onTestFailure(ITestResult result): Invoked each time a test fails.
onTestSkipped(ITestResult result): Invoked each time a test is skipped
onTestStart(ITestResult result): Invoked each time before a test will be invoked.
onTestSuccess(ITestResult result): Invoked each time a test succeeds.

IInvokedMethodListener: The working of this listener is also exactly the same as ISuiteListerner & ITestListerner and the only difference is that it makes the call before and after every Method. It has only two methods in it.
afterInvocattion(): Invoke after each method
beforeInvocation(): Invoke before each method
Ok, so now am guessing that you must be pretty aware of what listeners are all about and we also saw how to write a listener. Now comes the big question.
How to do it…
1) Create a ‘New Class’ file and give it a name ‘Listener‘, by right click on the Package and select New > Class.
2) Now Implements ISuiteListener, ITestListener and IInvokedMethodListener to this newly created class. For implementing a listener class, the class has to implement the org.testng.ITestListener interface. These classes are notified at runtime by TestNG when the test starts, finishes, fails, skips, or passes.
3) Create a ‘New Class’ file and give it a name ‘TestListener‘, by right click on the Package and select New > Class.

How do I let TestNG know that I have such a listener which it should invoke when it is executing my tests ?
There are essentially two ways of adding up a listener to a particular class.
1) Implement TestNG Listener to your test class

2) Listener tag in TestNG xml: Although approach 1 is more than enough to get you started, it’s not an “elegant” way of using Listeners, because you are forced to add this @Listeners section to each of your classes, which you perhaps won’t want. So what you do is, you create a TestNG Suite xml and then add up the listeners section to this suite xml file. That way, all of your tests would essentially leverage the listener that you wrote.

Output of the test case will look like this:


TestNg-Listener-0

 What is TestNG? And whats the difference between Junit and TestNG


TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:
  • Annotations.
  • Flexible test configuration.
  • Support for data-driven testing (with @DataProvider).
  • Support for parameters.
  • Allows distribution of tests on slave machines.
  • Powerful execution model (no more TestSuite).
  • Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc...).
  • Dependent methods for application server testing.
If you want to learn TestNG you can download it and see its documentation at “http://testng.org/doc/index.html”.
TestNG is extensively used in Selenium automation and Code testing instead of Junit due to its advantages over it.

Using TestNG Selenium automation becomes simplified and also the Parallel execution using Selenium Grid becomes easy.

TestNG have a lot of features in it and hence I wont be covering every thing in my blog. But will only talk about advantages of TestNG over Junit. I will be discussing about different ways of using TestNg functions in my coming blogs.

Advantages of TestNG over Junit
  1. In Junit we have to declare @BeforeClass and @AfterClass which is a constraint where as in TestNG there is no constraint like this.
  2. Additional Levels of setUp/tearDown level are available in TestNG like @Before/AfterSuite,@Before/AfterTest and @Before/AfterGroup
  3. No Need to extend any class in TestNG.
  4. There is no method name constraint in TestNG as in Junit. You can give any name to the test methods in TestNG
  5. In TestNG we can tell the test that one method is dependent on another method where as in Junit this is not possible. In Junit each test is independent of another test.
  6. Grouping of testcases is available in TestNG where as the same is not available in Junit.
  7. Execution can be done based on Groups. For ex. If you have defined many cases and segregated them by defining 2 groups as Sanity and Regression. Then if you only want to execute the “Sanity” cases then just tell TestNG to execute the “Sanity” and TestNG will automatically execute the cases belonging to the “Sanity” group.
  8. Also using TestNG your selenium testcase execution can be done in parallel.
--------------------------------------------------------




JUnit 4 and TestNG are both very popular unit test framework in Java. Both frameworks look very similar in functionality. Which one is better? Which unit test framework should I use in Java project?
Here I did a feature comparison between JUnit 4 and TestNG.
junit-vs-testngjpg

1. Annotation Support

The annotation supports are implemented in both JUnit 4 and TestNG look similar.
FeatureJUnit 4TestNG
test annotation@Test@Test
run before all tests in this suite have run–@BeforeSuite
run after all tests in this suite have run–@AfterSuite
run before the test–@BeforeTest
run after the test–@AfterTest
run before the first test method that belongs to any of these groups is invoked–@BeforeGroups
run after the last test method that belongs to any of these groups is invoked–@AfterGroups
run before the first test method in the current class is invoked@BeforeClass@BeforeClass
run after all the test methods in the current class have been run@AfterClass@AfterClass
run before each test method@Before@BeforeMethod
run after each test method@After@AfterMethod
ignore test@ignore@Test(enbale=false)
expected exception@Test(expected = ArithmeticException.class)@Test(expectedExceptions = ArithmeticException.class)
timeout@Test(timeout = 1000)@Test(timeout = 1000)
The main annotation differences between JUnit4 and TestNG are
1. In JUnit 4, we have to declare “@BeforeClass” and “@AfterClass” method as static method. TestNG is more flexible in method declaration, it does not have this constraints.
2. 3 additional setUp/tearDown level: suite and group (@Before/AfterSuite, @Before/AfterTest, @Before/AfterGroup). See more detail here.
JUnit 4 
    @BeforeClass
    public static void oneTimeSetUp() {
        // one-time initialization code   
     System.out.println("@BeforeClass - oneTimeSetUp");
    }
TestNG
    @BeforeClass
    public void oneTimeSetUp() {
        // one-time initialization code   
     System.out.println("@BeforeClass - oneTimeSetUp");
}
In JUnit 4, the annotation naming convention is a bit confusing, e.g “Before”, “After” and “Expected”, we do not really understand what is “Before” and “After” do, and what we “Expected” from test method? TestiNG is easier to understand, it uses “BeforeMethod”, “AfterMethod” and “ExpectedException” instead.

2. Exception Test

The “exception testing” means what exception throws from the unit test, this feature is implemented in both JUnit 4 and TestNG.
JUnit 4
      @Test(expected = ArithmeticException.class)  
 public void divisionWithException() {  
   int i = 1/0;
 }
TestNG
      @Test(expectedExceptions = ArithmeticException.class)  
 public void divisionWithException() {  
   int i = 1/0;
 }



3. Ignore Test

The “Ignored” means whether it should ignore the unit test, this feature is implemented in both JUnit 4 and TestNG .

JUnit 4






        @Ignore("Not Ready to Run")  
 @Test
 public void divisionWithException() {  
   System.out.println("Method is not ready yet");
 }





TestNG






 @Test(enabled=false)
 public void divisionWithException() {  
   System.out.println("Method is not ready yet");
 }







4. Time Test

The
 “Time Test” means if an unit test takes longer than the specified 
number of milliseconds to run, the test will terminated and mark as 
fails, this feature is implemented in both JUnit 4 and TestNG .

JUnit 4






        @Test(timeout = 1000)  
 public void infinity() {  
  while (true);  
 }





TestNG






 @Test(timeOut = 1000)  
 public void infinity() {  
  while (true);  
 }







5. Suite Test

The
 “Suite Test” means bundle a few unit test and run it together.  This 
feature is implemented in both JUnit 4 and TestNG. However both are 
using very different method to implement it.

JUnit 4 

The
 “@RunWith” and “@Suite” are use to run the suite test. The below class 
means both unit test “JunitTest1” and “JunitTest2” run together after 
JunitTest5 executed. All the declaration is define inside the class.






@RunWith(Suite.class)
@Suite.SuiteClasses({
        JunitTest1.class,
        JunitTest2.class
})
public class JunitTest5 {
}





TestNG

XML
 file is use to run the suite test. The below XML file means both unit 
test “TestNGTest1” and “TestNGTest2” will run it together.






<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >
<suite name="My test suite">
  <test name="testing">
    <classes>
       <class name="com.fsecure.demo.testng.TestNGTest1" />
       <class name="com.fsecure.demo.testng.TestNGTest2" />
    </classes>
  </test>
</suite>





TestNG
 can do more than bundle class testing, it can bundle method testing as 
well. With TestNG unique “Grouping” concept, every method is tie to a 
group, it can categorize tests according to features.  For example,

Here is a class with four methods, three groups (method1, method2 and method3)






        @Test(groups="method1")
 public void testingMethod1() {  
   System.out.println("Method - testingMethod1()");
 }  
 
 @Test(groups="method2")
 public void testingMethod2() {  
  System.out.println("Method - testingMethod2()");
 }  
 
 @Test(groups="method1")
 public void testingMethod1_1() {  
  System.out.println("Method - testingMethod1_1()");
 }  
 
 @Test(groups="method4")
 public void testingMethod4() {  
  System.out.println("Method - testingMethod4()");
 }





With the following XML file, we can execute the unit test with group “method1” only.






<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >
<suite name="My test suite">
  <test name="testing">
   <groups>
      <run>
        <include name="method1"/>
      </run>
    </groups>
    <classes>
       <class name="com.fsecure.demo.testng.TestNGTest5_2_0" />
    </classes>
  </test>
</suite>





With
 “Grouping” test concept, the integration test possibility is unlimited.
 For example, we can only test the “DatabaseFuntion” group from all of 
the unit test classes.



6. Parameterized Test

The 
“Parameterized Test” means vary parameter value for unit test.  This 
feature is implemented in both JUnit 4 and TestNG. However both are 
using very different method to implement it.

JUnit 4

The
 “@RunWith” and “@Parameter” is use to provide parameter value for unit 
test, @Parameters have to return List[], and the parameter will pass 
into class constructor as argument.






@RunWith(value = Parameterized.class)
public class JunitTest6 {
 
  private int number;
 
  public JunitTest6(int number) {
     this.number = number;
  }
 
  @Parameters
  public static Collection<Object[]> data() {
    Object[][] data = new Object[][] { { 1 }, { 2 }, { 3 }, { 4 } };
    return Arrays.asList(data);
  }
 
  @Test
  public void pushTest() {
    System.out.println("Parameterized Number is : " + number);
  }
}





It has many 
limitations here; we have to follow the “JUnit” way to declare the 
parameter, and the parameter has to pass into constructor in order to 
initialize the class member as parameter value for testing. The return 
type of parameter class is “List []”, data has been limited to String or
 a primitive value for testing.

TestNG

XML file or “@DataProvider” is use to provide vary parameter for testing.

XML file for parameterized test.

 Only “@Parameters” declares in method which needs parameter for 
testing, the parametric data will provide in TestNG’s XML configuration 
files. By doing this, we can reuse a single test case with different 
data sets and even get different results. In addition, even end user, QA
 or QE can provide their own data in XML file for testing.

Unit Test






      public class TestNGTest6_1_0 {
 
    @Test
    @Parameters(value="number")
    public void parameterIntTest(int number) {
       System.out.println("Parameterized Number is : " + number);
    }
 
      }





XML File






<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >
<suite name="My test suite">
  <test name="testing">
 
    <parameter name="number" value="2"/>  
 
    <classes>
       <class name="com.fsecure.demo.testng.TestNGTest6_0" />
    </classes>
  </test>
</suite>





@DataProvider for parameterized test.

While
 pulling data values into an XML file can be quite handy, tests 
occasionally require complex types, which can’t be represented as a 
String or a primitive value. TestNG handles this scenario with its 
@DataProvider annotation, which facilitates the mapping of complex 
parameter types to a test method.

@DataProvider for Vector, String or Integer as parameter






        @Test(dataProvider = "Data-Provider-Function")
 public void parameterIntTest(Class clzz, String[] number) {
    System.out.println("Parameterized Number is : " + number[0]);
    System.out.println("Parameterized Number is : " + number[1]);
 }
 
 //This function will provide the patameter data
 @DataProvider(name = "Data-Provider-Function")
 public Object[][] parameterIntTestProvider() {
  return new Object[][]{
       {Vector.class, new String[] {"java.util.AbstractList", 
"java.util.AbstractCollection"}},
       {String.class, new String[] {"1", "2"}},
       {Integer.class, new String[] {"1", "2"}}
      };
 }





@DataProvider for object as parameter
 P.S  “TestNGTest6_3_0” is an simple object with just get set method for demo.






        @Test(dataProvider = "Data-Provider-Function")
 public void parameterIntTest(TestNGTest6_3_0 clzz) {
    System.out.println("Parameterized Number is : " + clzz.getMsg());
    System.out.println("Parameterized Number is : " + clzz.getNumber());
 }
 
 //This function will provide the patameter data
 @DataProvider(name = "Data-Provider-Function")
 public Object[][] parameterIntTestProvider() {
 
  TestNGTest6_3_0 obj = new TestNGTest6_3_0();
  obj.setMsg("Hello");
  obj.setNumber(123);
 
  return new Object[][]{
       {obj}
  };
 }





TestNG’s 
parameterized test is very user friendly and flexible (either in XML 
file or inside the class). It can support many complex data type as 
parameter value and the possibility is unlimited. As example above, we 
even can pass in our own object (TestNGTest6_3_0) for parameterized test



7. Dependency Test

The
 “Parameterized Test” means methods are test base on dependency, which 
will execute before a desired method. If the dependent method fails, 
then all subsequent tests will be skipped, not marked as failed.

JUnit 4

JUnit framework is focus on test isolation; it did not support this feature at the moment.

TestNG

It use “dependOnMethods “ to implement the dependency testing  as following






        @Test
 public void method1() {
    System.out.println("This is method 1");
 }
 
 @Test(dependsOnMethods={"method1"})
 public void method2() {
  System.out.println("This is method 2");
 }





The “method2()” will execute only if “method1()” is run successfully, else “method2()” will skip the test.



Conclusion

After
 go thought all the features comparison, i suggest to use TestNG as core
 unit test framework for Java project, because TestNG is more advance in
 parameterize testing, dependency testing and suite testing (Grouping 
concept). TestNG is meant for high-level testing and complex integration
 test. Its flexibility is especially useful with large test suites. In 
addition, TestNG also cover the entire core JUnit4 functionality. It’s 
just no reason for me to use JUnit anymore.



References

TestNG
 ————
 http://en.wikipedia.org/wiki/TestNG
 http://www.ibm.com/developerworks/java/library/j-testng/
 http://testng.org/doc/index.html
 http://beust.com/weblog/

JUnit
 ———–
 http://en.wikipedia.org/wiki/JUnit
 http://www.ibm.com/developerworks/java/library/j-junit4.html
 http://junit.sourceforge.net/doc/faq/faq.htm
 http://www.devx.com/Java/Article/31983/0/page/3
 http://ourcraft.wordpress.com/2008/08/27/writing-a-parameterized-junit-test/

TestNG VS JUnit
 ——————
 http://docs.codehaus.org/display/XPR/Migration+to+JUnit4+or+TestNG
 http://www.ibm.com/developerworks/java/library/j-cq08296/index.html
 http://www.cavdar.net/2008/07/21/junit-4-in-60-seconds/
Posted by at