Q&A with the CEO


Q: What is BIQ?
A: One of our customers summed it up: BIQ takes OLAP data analysis, an incredibly useful business tool, out of the hands of "experts," and puts it into the hands of ordinary business users.

BIQ can save you money immediately, while far outperforming much more expensive spend analysis products.

BIQ can run self-contained on a desktop or laptop computer — or on a central server. BIQ requires no special data processing knowledge or skill, other than ordinary competence with Microsoft Office tools. Users can load transaction data into the system by themselves — many customers prefer not to involve their IT departments at all — and then enhance the data by defining dimensions, hierarchies, and mapping rules with point-and-click screens. The resulting dataset can then be analyzed with BIQ's powerful Viewer and its suite of analytical tools.
Q: What's the difference between BIQ and ERP systems, BI systems, and other data warehouse tools?
A: The key difference is that BIQ's data organization can change in real time. Ad hoc data analysis requires the ability to change hierarchies, dimensions, and mapping rules quickly and easily. A fixed schema means a fixed and limited set of exploration capabilities, quickly outgrown by any analysis organization. ERP and BI tools are data warehouses, shared by (sometimes) hundreds of users. It's not practical to consider changing the data organization for an ad hoc report, even if it were technically possible.

With BIQ, non-technical people can perform complex data manipulation all by themselves, simply and easily. It's pretty obvious: if you can't manipulate the data structure, you can't do ad hoc analysis. That's why people are still building reports by hand, rather than leveraging powerful technologies like OLAP and data mapping. Conventional tools can only show you what they're set up to show you.

The problem is that some IT department or team of analysts has to set up the schema and the data relationships. Then you have to live with what's been set up, usually for long periods of time. Unfortunately the whole thing breaks down in the first five minutes when you try to do real-world analysis. That's because no matter how clever your set-up team is, it just isn't possible to anticipate in advance all the ways that you'll need to see your data.

I've been in meetings where users can't agree on a commodity structure or even how the existing cost centers roll up. So how is some group of IT people or analysts going to resolve this, and make everyone happy with the decision? It's a catch-22. You can't share the schema, because then you can't change it. A tool can't be both a data warehouse and an analysis package. Any changes you make to the schema will lag demand by weeks, and by the time the changes are made, they'll be obsolete again. Which means you can't use the product for ad hoc analysis, unless you get very lucky indeed, and the schema just happens to support what you're asking.
Q: Does BIQ tie directly into accounting or ERP systems?
A: No. BIQ imports transactions in .csv (comma-separated variable) format through its powerful built-in Data Loader. It then "plays" those transactions through its comprehensive rules system to cleanse them and alter them, and only then allows them into the dataset. The Data Loader is able to knit hopelessly disorganized data files into a coherent and useful OLAP dataset. In fact, it's so powerful that some of our customers use it to prepare data for so-called "enterprise" products.

Analytical systems should always be at arms length from the accounting system, because otherwise they are severely limited by that system's data organization. BI and ERP-based analysis systems generally tie directly into the accounting system, and therefore live and die by the accounting system's organization. Hence the oft-repeated slam against BI tools: "All you can see is what's in the accounting system, anyway."

Because BIQ isn't tied to the accounting system, it can build different views of accounting data, integrate data from other sources, and construct completely different hierarchies. One of our customers maintained several cost center dimensions for many months, mirroring a disagreement between senior management as to the correct rollup method!

Finally, BIQ differs most profoundly from ordinary BI tools in its ability to extract information that's normally invisible. In the case of spend analysis, BIQ's rules mapping system can be used to build a true Commodity dimension — a dimension describing the commodities actually purchased, not just the GL code or Vendor, which are at best poor approximations to Commodity, and at worst nearly useless. The rules system can then be leveraged to build a Contract Spend dimension, a function of Commodity, Vendor, Time, and Geography. This is very different indeed than merely tacking on a pretty report writer to an existing system.
Q: Isn't loading data the big "catch" to these systems?
A: It often can be. So that was the first problem we solved.

First, we built a facility that takes the magic out of loading data. Our Data Loader can turn an arbitrary collection of barely-related files into a coherent, structured dataset with ease. Second, we built tools to enable point-and-click creation of dimensions and measures, along with the construction and manipulation of data hierarchies. These tools include a highly-sophisticated search engine and our unique Dimension Editor that, working together, make this process a breeze. Third, we built a data mapping engine that one customer described as "the simplest interface for data mapping that I could imagine." With point-and-click simplicity, you can create and map new dimensions easily and quickly.

The key was to make all these facilities accessible to business computer users. We succeeded.

The proof is in the pudding. We have customers — ordinary business computer users — routinely loading, building, and analyzing complex spend analysis datasets in incredibly short periods of time, all by themselves. No IT assistance; no ten guys in Bangalore "helping out"; no hassle with consultants or outside services contracts.
Q: What does BIQ do that's new or better?
A: We do lots of very new things, like reference queries, dimensional measures, treemaps, and Excel model population. I could stop right there, because nobody else does all these things. But, if I were to really drill down on the difference, it's that BIQ is a tool rather than a pretty picture that you can't change. It's a tool that encourages experimentation. Go ahead, try out a new organization. Create a new dimension. Map some transactions in a different way. Build a crosstab, throw it away, build another one. Build and populate an Excel model; then change the drillpoint and re-populate the model.

A BIQ user learns more about her data in a half day than she used to learn in weeks. There's no need to wait for anyone -- no need to wait for IT personnel, or for some vendor to make a mapping change, or for someone else to "republish" a dataset overnight. The user can change the underlying data schema at any time. She can create new dimensions, change hierarchies, or map transactions around from place to place. And, she can do this in real time — literally in seconds. Then she can apply the full power of the BIQ analytics engine to her problem, and get the answer immediately, just as though a team of analysts had adapted the schema specifically to her requirements. It's instant gratification.
Q: There are a lot of analytics systems. Why build another?
A: We had three key insights.
  • First, we realized that conventional enterprise-wide analytics systems aren't used for analytics at all, at least not the ad hoc analysis that one would imagine. Instead, they are used as simple data warehouses, with standardized reports and views. The reason is that their rate of change is too slow to support ad hoc analyses, many of which require structural changes to the data organization. We invented technology to solve that.
  • Second, we believed that we could construct a data loading and translation facility that is both usable by non-programmers, and integrated right into the product. We've had customers with zero IT training build entire datasets on their own, validating this many times over.
  • Third, we realized that for many enterprises, the notion of paying large sums for data cleansing or other data services simply isn't cost-effective or worthwhile. End-users have sufficient information to do these tasks themselves, incrementally over time, armed with the right tools. So we built those tools.

There were also three practical considerations. First, we felt that a powerful spend analytics solution could be delivered on the desktop for a fraction of the cost of a hosted solution, and that this would open our potential market far beyond the Fortune 1000.

Second, we believe strongly in empowering non-technical people, and in getting techies out of the loop whenever possible. It shouldn't require an IT department or some vendor on long term contract to maintain or support a financial analysis program.

Third, we felt that by properly leveraging other desktop products, specifically Microsoft Excel, that BIQ could provide synergies that web-based products could never achieve.

The result is that with BIQ, we can deliver a solution that's cost-effective, delivers value immediately, and makes heroes out of financial analysts.
Q: What about a web-based product?
A: With the advent of multi-core 64-bit machines from AMD and Intel, we perceived over 18 months ago an opportunity to allow BIQ to grow to extremely large dataset sizes (150M+ transactions) on that hardware. However, we knew that it would be a while yet before ordinary PC's emerge with that kind of capability. Therefore, we saw a need for a server-based solution that would allow multiple users to share expensive hardware.

We released our new 64-bit-capable, client-server-capable product in January 2005. Our client-server product is designed to run behind the firewall on an ordinary desktop server, and can also be deployed to the web. But — unlike any other system — our client-server offering enables individual power users to manipulate their own cubes, independent of others, just as BIQ desktop users do today. We strongly believe that the ability to manipulate dataset structure in real time is essential for deep analytics. A shared, unchangeable dataset is far less useful, and by itself offers no help to the serious analyst.

Regardless of our client-server capability, we continue to feel strongly that BIQ's ability to install on an ordinary desktop machine, independent of servers of any kind, is critical to our success. Our consulting customers would agree — it's enormously useful to be able to analyze billion-dollar datasets from an airplane seat! Our new product, which includes both client-server and desktop/laptop capability in the same package, gives us the flexibility to service virtually any customer, and to convert any customer instantly from one deployment model to another. In addition, our unique peer-to-peer capability, in which any BIQ-installed PC can be both client and server, allows groups of analysts to collaborate in very new and interesting ways.
Q: You built your own OLAP engine. Why not use a commercial product?
A: I have to get a little technical to answer this. There are three important lessons you learn by going down that road. Lesson number one is that most commercial OLAP databases work by pre-calculating joins. This means that you have to "publish" new datasets with a lengthy offline process. If an error is made, you have to rewind the clock and start over. What happens in practice is that publish cycles are almost always late, because someone always forgets or screws up something.

Lesson number 2 is that commercial OLAP databases can't pre-calculate datasets that have a certain shape or organization. If the branching factor is too large, the pre-calculation step may never finish. I watched one commercial OLAP database spin for over a week without finishing, on a dataset 1/3 as large as the datasets we routinely run with BIQ. Other commercial OLAP databases state right up front that they have branching factor limitations. Usually, limitations like that mean that extra levels of the hierarchy have to be "invented" in order to make the OLAP system happy. That makes it hard to use an analysis system, because visibility is narrowed artificially.

Lesson number 3 is that OLAP databases are designed to drill on one dimension at a time. If, as with BIQ, the intent is to show the effect of a drill on all dimensions simultaneously, the OLAP database must be called N times, once for each of the N dimensions. BIQ's rollup engine is carefully optimized to roll up all dimensions simultaneously, saving millions of CPU cycles (and providing much better response times).
Q: How do multiple users of BIQ within the same company coordinate their activity?
A: In client-server mode, the answer is obvious: someone takes responsibility for publishing a canonical copy of a dataset to a server.

In our laptop/desktop, or peer-to-peer deployment models, customers typically designate a particular dataset as a reference, and publish it to a central server as a starting point. It's just a couple of mouse-clicks to copy dimensions or even entire datasets. So users typically make a copy of a dimension or dataset, remap or re-family it, and operate on the copy. And, if they ever get into trouble, they can re-load the reference dataset from the server.

BIQ also provides collaboration features, such that users can share work on the same project. For example, if one user makes a series of mapping changes to dimension X, she can export those changes to another user, who can then import them into his dataset. Any conflicts are either automatically reconciled by the system, or reported on-screen for point-and-click resolution.

BIQ also provides automated support for building sub-datasets, suitable for distribution to organizations that may not need to see all of the data. These datasets contain only the designated transactions, and no others.

It's pretty obvious: if you can't manipulate the data structure, you can't do ad hoc analysis.