Congratulations, you’ve made the choice to embed a BI solution into your product line. You have thought through the implications of building it yourself and have decided you want to focus your team on what makes you money, your specific domain. Unless you sell a BI solution, you don’t want to keep up with the BI feature race. As we’ve talked about earlier, that market is crowded, and always moving.
But, that same benefit that comes with pulling in a third-party BI solution also can cause some headaches. What if one of those new features broke existing reports? Worse yet, what if they broke a report that your customer created that you are not even aware of? That won’t get caught in your regression test suite. Add onto this the potential breakage from underlying schema changes, and you have a very uncomfortable upgrade cycle.
As you would expect, this calls out to some type of automation. Manually testing all your reporting content would be overwhelming. Sure, you can write UI tests, but that is a trying ongoing process.
This is where using a solution that is set up for embedding can help you. If the solution facilitates Single Sign On, and has an API you can use to execute your reports, you are in business.
We are going to walk through automating report testing for a Yellowfin BI deployment. The example case describes a stub for testing against a SaaS deployment.
What needs to be accomplished:
- Set up a test environment that mirrors production, including users and source data.
- Depending on resources, set up a sampling of users and reports to execute, or run through every user and report. This is entirely dependent on your user and report counts.
- Enable Single Sign On authentication in your solution. If you are already doing it, you know exactly how it works. If you are not, your automation will use it to authenticate in as a user.
- Pull the user and report lists to test.
- Authenticate and execute reports
Depending on the size of the deployment, you may build out a smoke test that executes every report for every user. In the following example, we’ll cycle through all the steps within a deployment, and execute every report to confirm it can successfully run.
The following code snippets assume:
- A System Administrator will be executing the API calls
- SIMPLE_AUTHENTICATION has been set to TRUE in the Yellowfin Configuration data, Configuration table.
The second step insures that we do not need to know the password for each user, as the administrator is able to execute reports on their behalf.
The first step is creating the Web Service objects we’ll be using to call into the Yellowfin deployment. In this case, I’m using their Java SOAP API.
We’ll use the AdministrationServiceClient Java helper object to pull users and identify the reports for each user.
AdministrationServiceClient asc = new AdministrationServiceClient(hostname, port, username, password, adminClientURL);
We’ll directly generate the ReportService Soap Binding as it provides more functionality to execute reports.
ReportServiceService s_rpt = new ReportServiceServiceLocator(hostname, port, reportClientURL, false); ReportServiceSoapBindingStub rssbs_report = (ReportServiceSoapBindingStub) s_rpt.getReportService(); ReportServiceResponse rs = null;
Now that we have our Web Service objects created, we can proceed.
Next, we’ll pull the list of users on the deployment to be tested:
HashMap<Integer, AdministrationPerson> users = new HashMap<Integer, AdministrationPerson>();
AdministrationGroup[] groups = asc.listGroups();
for (AdministrationGroup administrationGroup : groups) {
AdministrationGroupMember[] members = administrationGroup.getGroupMembers();
for (AdministrationGroupMember member : members) {
users.put(member.getInternalId(), asc.getUserByIp(member.getInternalId()));
Once we have the list of users, we can cycle through them, identifying all the reports they have access to, then executing every report. With a more complete test, we could run through each Client Organization for the user and execute all the reports visible for that user in each Org. This simple test just does that against the first Org.
Iterator<Integer> userIterator = users.keySet().iterator();
while (userIterator.hasNext()) {
AdministrationPerson person = users.get(userIterator.next());
AdministrationClientOrg[] orgs = asc.getUserAccess(person);
String clientRefId = null;
if ((null != orgs) && (orgs.length > 0)) {
clientRefId = orgs[0].getClientReferenceId();
}
asc.setClientReferenceId(clientRefId);
AdministrationReport[] ar1 = asc.listAllReportsForUser(person);
System.out.println(person.getEmailAddress() + " | " +
person.getFirstName() + " " +
person.getLastName());
At this point, we iterate through all of the reports for the user and execute. Here we are exporting using PDF format.
for (AdministrationReport report : ar1) {
ReportServiceRequest rReq = new ReportServiceRequest();
rReq.setLoginId(username);
rReq.setPassword(password);
rReq.setOrgId(1);
rReq.setReportUserId(person.getUserId());
rReq.setReportId(report.getReportId());
rReq.setReportRequest("PDF");
rReq.setReportClientReferenceId(clientRefId);
rs = rssbs_report.remoteReportCall(rReq);
System.out.println(report.getReportName() + " " + rs.getStatusCode());
Given the standard Yellowfin tutorial database, the output would look like:
admin@yellowfin.com.au | System Administrator Athlete Age Group Breakdown SUCCESS Athlete Cost Summary SUCCESS Athlete Demographic Breakdown SUCCESS Athlete Demographic Spread SUCCESS Athlete Figures by Location SUCCESS
This will allow you to run through every user and every report, and confirm the reports are executing successfully. This example will not highlight problems in layout, but it will identify failures caused by schema changes or reports that were not upgraded successfully.
