Java Data Warehousing

Adept Developer was the software partner for Wysk from 2012 to 2015 when Wysk ceased business operations. Wysk aggregated data for businesses such as federal and state business filings, tax documents, real estate, litigation, trademarks and patents.

Adept Developer was responsible for two major components: data acquisition and the public facing website and search engine. We built a large scale database system for a variety of needs using Microsoft SQL Server and in some cases MariaDB. We built a sophisticated system for scheduling and importing data from a variety of file formats including: csv, excel, fixed width files, multiple document fixed with files, xml, json and web APIs.

Wysk was the first project that Adept Developer took on Big Data--dealing with literally trillions of records. We utilized a software program built on the Eclipse platform called Talend. Talend specializes in ETL transformations. We wrote different Talend jobs to take in a variety of file formats and normalize them to a standard format within SQL Server.

Technologies Used

  • ColdFusion
  • Railo
  • Microsoft SQL Server
  • MariaDB
  • Talend

Wysk Website

The public website for Wysk was built using ColdFusion, Solr and Microsoft SQL Server. ColdFusion integrates with Solr full text search engine out of the box, but for the massive needs of Wysk, we wrote our own wrapper so that we could utilize the latest version of Solr and features that weren’t exposed through the built-in Solr service.

Users could search for any business in the US by name, address and officer name. Once the appropriate business was located, they could order and view reports for that business that included: Secretary of State data, Franchise Tax information, Sales Tax information, Trademarks, Patents, Real Estate owned and Litigation information.

Integrations

Lexis Nexis Integration

Lexis Nexis CourtLink

We integrated with Lexis Nexis’ CourtLink product to run real time litigation searches based on company name. We integrated with the CourtLink’s SOAP-based web service to request and display results for our top of the line report.

DataQuick Real Estate Integration

DataQuick

We integrated with DataQuick’s real time real estate search engine to provide a list of properties owned by companies in our reports.

PostMark API Integration

PostMark

We integrated with PostMark’s API using ColdFusion to send reliable and secure e-mails to customers when their reports were ready to be downloaded.