So far in Part 1 of our journey, we have examined the pressing problems facing the MLS industry.
In Part 2, we identified the source of those problems.
In Part 3 we proposed a solution based on a totally new architecture of the core MLS technology, the database that drives the entire interconnected system. Now let’s look at how we get to this solution.

The road map to success for such a dramatic change in the infrastructure of our industry will be fraught with peril and filled with potholes. But with a common purpose and diligent attention we can achieve success.

Throughout this project I have been encouraged by the open-mindedness of most of the major technology companies I have talked with in pursuit of this goal. But such a pursuit closely resembles the traditional chicken-and-egg paradox – which comes first?

Does a major MLS vendor take the first step, segregate its database and open it to developers with an invitation to cooperate in a larger pursuit? This has been attempted on a limited scale and has seen some modest success. FBS, itself a major MLS technology provider, introduced its Spark Platform some years ago but it has not captured the imagination of the vendor community at large.

The Clareity Store seeks to address the interoperability challenges between MLS systems through a common authentication linkage between systems and vendors but does not take on the challenge of common interfaces across all database systems.

No single legacy vendor has dared to take the first step of divorcing the database from the applications layer and opening the system to all developers without licensing costs. At the same time, no group of application developers has aggregated its product lines in hopes of enticing system vendors to open their systems. Short of the few inquiries I received initially, no MLS CEO has taken the bold step of actually notifying an MLS vendor that the MLS was aggressively pursuing this approach and would require the MLS vendor to cooperate or lose the market. No one wants to go first.

Now an industry consultant with no stake in the current MLS vendor field has proposed someone, some technology vendor somewhere, take the first step and create the open database into which a community of application providers can offer their products across a broad base of MLS operators. But that vendor will want to see a broader demand base of support before it would be willing to expend the time and effort needed to even reply to a request for proposal.

The key to cracking the eggs and hatching the chicks is to identify and induce a group of MLSs willing to take the first step and ask for the system to be developed. Fortunately, a couple of intrepid MLSs were willing to take that first step and with their committed inquiries in hand, I approached a number of technology vendors to see if I could find one willing to work on this project.

But I had some firm criteria that a database vendor needed to meet to be considered. An endeavor such as this could not be just two guys in a garage who thought they could do it. Here were my criteria:

• It had to be a substantial company (read that as having resources, both financial and personnel, to actually do this) with a track record of aggregating data and managing a large, scalable database ;
• It had to be a company with experience not just in handling MLS data but in integrating multiple other databases into a cohesive property-centric system; and
• It had to be a company with minimal industry “baggage” — not a company to which the first reaction of most would be, “OH, NO, not them.”

That last requirement eliminated a number of the most likely candidates including the national portals as well as most of the major system vendors and all but a couple of public records providers. In an undertaking like this one, if a company had both lovers and haters, the stridency of the haters would probably overwhelm any attempt at honest debate and careful, logical consideration.

Fortunately I finally identified one company that met all my criteria but which I thought would, for reasons that will be obvious shortly, not be able to even consider getting involved. That company was the Realtors Property Resource – RPR – the subsidiary of the National Association of REALTORS^(R) that has been successfully operating a national-scale property-centric aggregated database of real estate information for over five years.

RPR has a specific mission — to supply Realtor practitioners with technology tools and information that would allow them to further demonstrate their value to better serve their consumer clients by providing service that their clients could get nowhere else. Their product is spectacular. Those that use it love it and it has been steadily evolving and improving to the point where those who use it regularly feel they couldn’t do without it.

RPR was created through the acquisition of the Cyberhomes technical team and product base from Black Knight (at that time called LPS). With the team and the technology, RPR negotiated data licenses for many of the public records and other data sets that LPS had accumulated and continues to license that data today. RPR has direct data feed relationships with more than 600 MLSs, including 900 associations, representing over 865,000 REALTORS^® and has already built the technology to facilitate sharing those data sets across any number of MLSs that have separately agreed to do so.

In my initial meeting with RPR, Dale Ross (CEO), Marty Frame (President) and Jeff Young (Chief of Operations) expressed one significant reservation about even considering this project. If RPR were to jump in, it might be very misunderstood by the rank and file Realtor as an attempt at a power grab by NAR — trying to create a National MLS and put all the locals out of business. Admittedly, such a scenario would be incredibly stupid for RPR to attempt because (a) they had contracts with hundreds of MLSs that prohibit them from competing to provide MLS services, and (b) many of the Realtor associations rely on income from their MLSs to provide other services to their Realtor members. Even though such a move would be political suicide, that fact alone would not be enough to keep some people from thinking there might be an ulterior motive. Conspiracy theories would abound and proponents of this new technology would be shouted down before they could even explain the benefits involved.

But if I were to come to RPR and ask them to respond to specific MLSs who have specific needs and are asking for help from RPR to meet those needs, well that would be a different story. If RPR were asked to repurpose technology that NAR had already invested millions of dollars to develop, and in doing so could create an alternate revenue stream for the company, the executives of RPR would be derelict in their management responsibilities not to respond.

And so I did. With RPR’s endorsement (and, full disclosure, their agreement to cover my time and expenses in this undertaking), I began contacting MLSs that were identified as strong potential partners, led by executives and leadership who were open-minded and forward-looking, leaders who would hear me out without shouting me down at the first mention of something new, different, challenging, but ultimately rewarding for all involved.

And that’s where we are today. We have eight MLSs from all over the country of various sizes and configurations to whom I have made this presentation and who have submitted letters requesting a specific proposal, in writing, to create the framework for this new technology. The response was great enough that RPR had to assign the project an acronym — AMP^™ — the Advanced Multi-List Platform^™.

It is critical to note at this point the name of the project is Multi-List Platform, not Multi-List Service. To quell any initial heartburn, let’s dispense with the elephant in the room question right up front. AMP^™ is not designed or intended to replace an MLS – Multiple Listing SERVICE. It is designed to provide the technology that the Service uses as their core system – the database platform upon which the system is built and through which Service is offered. AMP^™ is not an attempt to create a national MLS (a concept that, to my thinking, wouldn’t work anyway, but that’s another blog post). AMP^™ requires no changes in rules, regulations, oversight, staffing, management, or governance. The business of the MLS and the people who manage that business remain the same – just the technology behind the scenes changes.

RPR is now working on proving the concept behind the theory, putting details to the general outline of this new advanced architectural platform. They will be working with the MLSs who have stepped up as the pioneers brave enough to journey into a new world of technological innovation.

What will be the result of all these efforts? How will the problems be solved and what can all parties expect as the beneficial result of this hard work? Let’s examine those questions.

The Benefits for all Players

Separating and opening the database will immediately make possible many new approaches to MLS operations, all of which will benefit the brokers, agents, the MLS, the MLS stakeholders, and the vendors who support all of them.

Vendors

For vendors, new markets will open into which they can sell their products and services. Gone will be the closed monolithic system where only the primary vendor in a market was permitted to offer services. With some slight modifications to conform to the data and schema standards and to use the APIs for access, the CMA module from an FBS system, the farming system developed by Paragon, the CRM system developed by Stratus, the prospecting and lead management system in Matrix all could run just fine on the new, open and accessible database. Third party, non-MLS system vendors would no longer be challenged to write new code for every MLS system that comes along or convert their products when the MLS changes system vendors. The database would be the same across all systems.

• By opening the MLS, technology vendors will find more open markets, more agents available to buy their products, and more opportunity to increase revenue with lower development and support costs for the MLS provider.
• Developers will also be able to get out of the business of aggregating MLS data in order to simply make their applications work.
• Product marketers would be able to accomplish MLS integration much faster and easier since the API would be the same across all MLSs. Developers could write the code once and use it with equal facility in all installations.
• Vendors would no longer be challenged to write new code for every MLS system or convert their products when the MLS changes system vendors or adds a new data field.
• Mobile app developers would no longer be challenged to build a different application for every market.

Agents and Brokers

Agents and Brokers will find a new world of applications that were previously not available to them, and those that were available can be offered in a more palatable manner.

The days of an MLS providing one tool for all subscribers because the only way to acquire it at a reasonable cost was to buy it in bulk will be a forgotten memory. With an open MLS system, the role of the MLS changes from being the reseller of a bulk purchased service to the licensor of such a service that is then purchased only by those who want and need it. The major complaint that agents were being forced to pay for services they didn’t want or need, would be eliminated.

Further, brokers with already developed infrastructures could embed any search client or other “widget” or “plug-in” directly into their company’s intranet thus further enhancing agent loyalty and brand awareness. Once the RETS standard for updating transactions is completed and published, those same broker intranets could initiate and maintain listing records directly in the MLS database through an embedded listing maintenance widget. This would maintain the data integrity of the MLS while directly addressing broker demands for renewed control over the listing distribution process using Upstream or a similar management system.

Agent Benefits

• Agents will have more choice in picking from a menu of applications that were previously not available or integrated with their MLS system.
• Agents will be free to mix and match tools from different vendors to suit their needs and work styles.
• Agents will only pay only for what each agent uses by subscribing to each of them individually.
• An agent’s data stays synced wherever she goes, as she works across different applications.
• Agents would find listing entry and update to be much easier and more streamlined. In a parcel-centric database with each physical property has only one record, thus eliminating duplicate or “refreshed” listings.

Broker Benefits

• Brokers would no longer compete with the MLS in offering tools to agents because brokers can select which MLS tools are offered to their agents.
• Brokers can now receive a single data feed or fully integrated applications from multiple MLSs in which they participate using the same platform.
• Brokers with already developed infrastructures could embed any search client or other “widget” or “plug-in” directly into their company’s intranet thus further enhancing agent loyalty and brand awareness.
• Brokerage intranets could initiate and maintain listing records directly in the MLS database through an embedded listing maintenance page or widget.
• Any brokerage that wants to build its own applications can do so easily as authorized by license. If a brokerage operates across multiple markets the broker apps will be complete and work the same in all markets.

MLS and its staff

MLS administrators will find it much easier to meet their responsibilities for keeping the MLS’s biggest asset – its data – safe from marauders. Through judicious and automated data licensing processes and effective monitoring systems, the MLS can enroll tech providers at a record rate and offer subscribers an expansive library of apps while still maintaining control over the data.

The APIs used will pass only the data needed at that moment by that application. With few exceptions, the days of vendors needing to download the entire MLS database to make their products work will be gone.

Staff and Committee work that in the past has focused on product review and selection can be refocused on more productive activities. No longer would MLS leaders need to evaluate competing products in order to choose the best one for the entire market. Under an open MLS concept, all products that met the licensing requirements and conformed to MLS data rules could be offered equally in the market. The end-user, not the MLS staff or volunteers, would make the buying decision.

MLS Benefits

• MLSs can easily bring products and services to their subscribers thus satisfying the demands of their brokers for choice on a platform that evolves in the future.
• The MLS staff can meet their responsibilities to monitor data access and use, and easily cut access to offenders.
• The APIs used will pass only the data needed at that moment by that application. With few exceptions, the days of vendors needing to download the entire MLS database to make their products work will be gone and with them the uncertainty about what the vendor would or could do with the data now in their possession.
• The MLS can eliminate nearly all costs for providing IDX/VOW, as well as removing the need to rely on an outside syndication company to distribute data.
• MLSs will no longer have to purchase tools in bulk because that was the only way to acquire them it at a reasonable cost.
• Since public records and tax data are integrated with property listings in a parcel-centric paradigm, licensing costs are reduced.
• Automated licensing processes allow the MLS to offer a broad library of apps, while maintaining strict control over the licensees.
• The simplified licensing process will allow the MLS to set up many more vendors more quickly, thereby providing more sales options and the opportunity for more revenue streams.
• By eliminating competition between proprietary MLS system vendors, there will be no more MLS conversions – ever. The nightmare of retraining the entire membership on a new system will end.
• Authorized applications do not need to be supported technically with data feeds, greatly reducing the support costs to maintain and manage those feeds.
• In the “pay only for what you use” scenario, site licenses would no longer be practical or desirable. Agents would subscribe to and pay for each piece of the system they want to use.
• MLSs will increase compliance and improve data integrity. A single property record, with nearly all of the required fields already resident, will eliminate most typographical errors since new data entry will be minimized.

Association owners

• For associations thinking about a future merger with another association, this structure offers much flexibility.
• Combining the two databases of merging MLSs on the same platform and using the same structure is a non-event.
• On the other hand, having a common database may negate the need to consider a merger. Overlapping market disorder is a thing of the past and with it went one of the more compelling reasons boards consider merging.
• All of the systems on the common database platform could maintain their local culture and integrity by simply sharing data with subscribers of neighboring MLSs.
• System sharing could be with or without an accompanying offer of cooperation at the option of each MLS. The association and MLS could still decide.

In the past year, RPR has introduced the concept of an open MLS platform directly to over 20 MLS chief executives and to industry leaders across the country through its RPR Advisory Council. The reaction has been overwhelmingly positive. AMP^™ provides RPR an opportunity to continue and expand the RPR mission of supplying Realtor practitioners with tools and information that increase their value to their clients, while at the same time supplying much-needed innovation through technology and infrastructure to the MLS community.

In Conclusion (please, hold your applause)

Is this a change in direction for RPR or just a logical extension of their current mission to be a national technology solution provider? The answer is that RPR’s AMP^™ project is the logical extension of the mission and purpose RPR is already fulfilling. It is an extended use of the formidable technology and infrastructure RPR has already developed.

I am hopeful that in the months to come RPR will be able to demonstrate that the ideas of the MLS tools of choice, of universal standardized data access, and of the menu of services approach to providing MLS applications to agents will prove to be a real possibility and eventually a reality. I look forward to continuing to work with RPR as we journey down this path. If you want to come along, please let me know. I’ll keep you updated on our progress.

–

If you would like a printed copy of this series,
please click on the cover page
to download the full content in PDF format.

In Part I of this series we looked at the problems and challenges facing the MLS community of today.
In Part II we defined the problems in greater detail and examined the systemic and institutional monopolies that perpetuate those problems.

In this installment we will look at a possible solution, one based in technology not in governance or management/ownership structure, one that presents benefits for all parties in the MLS ecosystem.

The Proposal

The climate is ripe for innovation in the MLS technology industry. Advances in data storage capacity, exponential increases in interconnection speed, the advent of the “app store” approach for mobile devices, and the growth in services that compete with the MLS for the agent’s attention and tech-spend all point to an urgent need to redefine the traditional vendor-MLS relationship.

The industry has long dreamed of a method whereby the MLS, and subsequently its subscribers, would not be hamstrung by the need to select a single vendor, one that controls not only the database but also all of the applications that use the database. Under the moniker of “front end of choice,” the industry has pursued standardized data definitions, query/response methods and transport protocols.

The concept of an open MLS that would allow any number of user interfaces (front ends) is not new. It was discussed as long as 15 years ago when I was still a novice product manager at Interealty Corp. (later to become part of CoreLogic after multiple ownership changes). The concept was revisited in 2008 when Saul Klein (then of Internet Crusade, later Point 2 and points beyond 2) published his MLS 5.0 manifesto. In his paper, Saul said the new MLS must be “open, collaborative, self-organizing and self-policed.” He added, “MLS needs to redefine itself from a purely business-to-business network tool to a marketing facilitator for its participants and subscribers. It needs to take advantage of its assets and shift its paradigm from (just) information about what is for sale to information on all property whether for sale or not.”

Seven years later, the concept remains unfulfilled. In the absence of advances in MLS technology, others have stepped in to fill the vacuum. National websites that combine listing data with property data, demographics, lifestyles, and user generated content and ratings have proliferated. But the MLS world now has an opportunity to catch up and take the lead in a big way. The stars are aligning and many pieces of the puzzle are starting to fall into place.

Data Standards – Finally

Progress has been slow, but lately great strides have been made by the Real Estate Standards Organization (RESO) in codifying data norms through the Real Estate Transaction Standard (RETS). RETS has been used for years to standardize the way data is distributed from the MLS to licensed recipients. Now, through adoption of the Data Dictionary and the pending release of the RETS-API, RESO is pushing the industry toward more internal standards and therefore more interoperability.

A few MLSs have gone so far as to embrace the RETS Update Transaction that will allow brokers to upload (and maintain) listing records into the MLS from the Broker’s intranet, rather than the MLS’s front-end interface. This transaction standard has made possible the much-anticipated Upstream project, which will (as best we understand it at the time of this publication) aggregate new listings from participating brokers and feed them TO the MLS (as well as to other syndication destinations) from the broker’s back office, rather than the reverse. Upstream has the potential to be the first, and most widely used, front end of choice for a new generation of MLSs willing to embrace it.

The nation’s largest MLS, California Regional (CRMLS), has already announced they are ready to receive the listing input feed from Upstream, using technology supplied by Atlanta based Bridge Interactive Group. The time is right to explore how much farther we can push the standards and how much closer we can get to achieving not just front end of choice but ALL tools of choice. A change in backend database architecture opens a whole world of possibilities.

The Open Database concept

In order to facilitate greater technological innovation, the industry needs to overhaul the current system starting with the database and get rid of the legacy of closed proprietary repositories. Technology has advanced to the point where keeping the database proprietary and inaccessible is no longer necessary for a well functioning integrated system.

By separating the database (the back end) from the other elements of the system (the front ends) and opening access to the database through a collection of application program interfaces (API) and software developer kits (SDK), by contributing the APIs to a common-good licensing system and the software code that drives them to an open source repository, access to and open use of the database would be available to any technology partner with whom the MLS enters into a license agreement.

In such an environment, the database remains the core of the system. But it is not inextricably linked to a fixed set of front-end applications, those tools that an agent uses to do business. Any tool, written by any developer, that follows the published guidelines for access and connectivity (the open APIs) could query for data and receive back, in real time, from a live database, exactly the data that it asked for and nothing more. Just think of the possibilities:

• Agents could choose from many different versions of core MLS functionality by picking from a catalog of Hotsheet apps, CMA apps, Buyers’ Tour apps, and on and on. Agents pay only for what they use, and in many cases (certainly not all) lower their monthly fees.
• Most of the broker complaints listed by The Realty Alliance would/could be addressed.
  • Products and services are unbundled and open to free selection
  • Products that compete with a broker’s service could be masked from agents in that firm
  • Data feeds would be standardized among all systems on the database
  • Broker back-office integration would be greatly simplified
  • Minimal downloading would mitigate most data piracy
• MLS conversions would never again be needed. Instead of changing the whole system to get new functionality for its subscribers, the MLS would simply license new application providers. The marketplace would sort out the good from the bad.
• Data downloading and synchronization would be a thing of the past.
• Data distribution management would be all but eliminated because except in a few large-scale cases the applications would not download all the data from the MLS.

The ramifications and possibilities that follow this initial step are myriad, complex and full of potential. We will discuss some of those later in Part IV of this series, but first we need to get past the initial obstacles to this concept.

Previous roadblocks to progress

Why has no one tried this before? There are many reasons, some large, some small, but all difficult to overcome.

Finances

No MLS vendor or MLS operator has the financial wherewithal (either resources or incentive) to create such a new ecosystem from scratch. Many are hard pressed to keep pace with the massive financial investments being made by the online portals that they see as a major competitive threat, let alone divert more funds to a speculative and potentially disruptive venture. The costs of such a start up, with no guarantees of a return on investment, are simply too large and too risky for any single entity to take on.

Politics

As with nearly everything else in the MLS business, politics play a major role. Vendors do not want to alienate their current customer base in the pursuit of larger opportunities. MLS executives are already fearful of job security in the face of growing pressures to consolidate and regionalize. Elected leadership is most comfortable keeping the ship on an even keel during their short term in office, rather than charting new waters in Oceania Incognito. No one wants to take the first step or even make the first suggestion of stepping out in a new direction.

In doing the research for this project, I spoke with many MLS leaders, both vendors and CEOs. Without exception, all were intrigued by the concept but almost every one of them on both sides of the equation expressed fear of going first, of even being the first to publicly express interest.

[cryout-pullquote align=”right” textalign=”right” width=”33%”]All (MLS CEOs I spoke with) were intrigued but no one wanted to go first. [/cryout-pullquote]

I also spoke with a number of technology companies that might be able to provide some of the pieces of the puzzle, the elements that would be necessary to complete the entire deconstruction and reassembly of a new system. All (except for one we will discuss shortly) that I spoke with felt they could make contributions. But like the incumbents, none of them wanted to be the first one tapped as a leading contributor.

Inertia

Objects and MLSs at rest remain at rest unless acted upon by an outside force. So said Sir Isaac Newton in talking about the first MLS. (Actually I think he was describing planetary orbits, but the concept is similar.)

MLS executives have plenty to worry about without some industry consultant coming along and telling them they should expand their horizons and think of a new way of structuring their technology base. The market is emerging from years of gloom and doom and as market activity increases so do the inquiries and complaints from competing practitioners. Rules violations have to be attended to. New complaints from large brokers about past MLS business policies come under fire. NAR considers new association charter requirements and the execs must determine the impact on their local membership.

But some proposals need to be considered, regardless of the noise competing for the executives’ attention. The future of their MLS and their industry rely on constantly moving forward and innovating, changing direction when needed and never remaining at rest. The inertia of the MLS of the past must at some point change or the MLS may perish.

The Proposal – What it is and how it works

There must be a better way. My research examined the current state of MLS technology and my conclusions posit a new technology structure that could address the current chaos to the benefit of all parties involved.

Such a proposal requires a reconsideration of the MLS at its very core – its underlying platform. I believe there is an opportunity to deconstruct the MLS, to separate user interface, business logic, and data layers of the platform into separate structures. Such deconstruction would enable the MLS to become far more open and flexible. It would create a framework for technological innovation and the opportunity for more choice by end users and vendors alike.

In order to open the architecture of the MLS, we must first break apart what has up until now been a closed, locked system. We need to split the database away from the applications that are used to access it, make it a standalone storage engine with all of the business rules but only the minimal structural rules and requirements needed to support real-time retrieval and management of the data.

At the same time we must create a tool set of Application Programming Interfaces (APIs) and Software Development Kits (SDKs) to allow access by any and all developers willing to conform to terms of a license and the established standard structure. Throughout this process we need to ensure that the database technology provider openly licenses the APIs through a common open-market system and makes them available to any and all interested parties at no charge. The primary database technology vendor must openly publish all the common source code needed by all applications through the APIs. These APIs allow development of a wide variety of new applications that can inter-operate against and across any database that is structured in the same open manner.

The deconstructed MLS also serves to minimize many of the conflicts between the parties in the industry today. In the process it would redefine the meaning of one of the continuing criticisms of the MLS – that it levels the playing field by charging large participants for the cost of supplying services and support to smaller ones. The new level field would present equal opportunities for all with equal access to the building blocks in a “pay only for what you use” economic model.

Such a dramatic shift in the technology landscape will not occur overnight. We will need to overcome the usual objections – this is too much; too new; you’re moving too quickly; not the way we’ve always done it. But baby steps now will help us reach the ultimate goal – not just front-end options but the MLS tool box of choice.

The philosophy behind the change

We anticipate not only a change in the technology used but a change in the philosophy behind an MLS database. Currently, MLS systems create a new data record for each and every listing contract entered into the system. This creates a considerable amount of duplicate data for record fields that do not change often (beds, baths) if ever (address, lot size). Most systems also auto-populate each new listing record with data stored in a public records (tax) system and do it again each and every time a property is listed. This increases the number of duplicate fields and the quantity of redundant data.

Property-centric database

The new system envisions a property-centric database structure with each physical property having one property record, whether or not it has ever been transacted through the MLS. When a broker signs a new listing agreement, 99% of the data needed to complete a traditional listing record will already be stored in that property record, subject to verification and update if needed.

To this property record, an agent needs to add only a few fields to “claim” the property (thank you Zillow!) and indicate that it is now actively for sale: list date (when the listing will show as for sale), Listing Contract Date (when the listing agreement was executed), List Price (asking price), the identity of the listing agent (which would be pre-populated based on the login of the person making the entry, but subject to modification if that agent is merely helping another), and the cooperative compensation offered to other brokers. Thus each new listing event becomes an edited change to the master property record, not a new record in and of itself.

At time of activation, the agent could, optionally, add new photos, a description of the property (remarks), private showing instructions to other agents, virtual tours, or any other marketing information needed. Later, the agent would be able to update the status of a listing as it moved through the sales process.

[cryout-pullquote align=”right” textalign=”right” width=”33%”]Each new listing becomes an edited change to the master property record, not a new record in and of itself.[/cryout-pullquote]

The history of any changes, additions, deletions of fields or data within the property record would be religiously maintained so that no data would ever be lost. It would always be available to the MLS administration for rules enforcement purposes and to authorized governmental oversight bodies, as required by law.

This system would practically eliminate the most frequent and most troubling “tricks” that agents try in an attempt to game the system for marketing advantage. Agents would no longer be able to deactivate a listing and add it again as a new record. There is only one record, with a history that shows such activity.

Agents would not be able to create duplicate listings for the same property in multiple areas or zip codes. There is only one record in one location. (There would be provision for multiple treatments of that property, for example showing it for sale and/or for rent at the same time.) And perhaps most important, having a single property record with 90% or more of the content standardized and unchanging from listing to listing would eliminate the vast majority of typographical errors since data entry will be minimized for each listing.

Overcoming initial fear

The first reaction from the MLS vendor community to such a proposal is likely to be fear and trepidation. Opening the database to “outsiders” would undermine the very foundation of vendors who have stood the test of time immemorial – or at least since the late ‘80s when the computerized MLS network became a reality. Vendors won’t stand for that.

Or will they? What if we could show that an open system could actually expand market opportunities for vendors, increase their customer base, and make them more money by providing more of their applications than the current system, and not only in those market areas that are under their “control”?

The model being proposed here is not unlike that of the app store developed by Apple and mimicked by Google’s Android marketplace, where an open operating system is available to any and all developers, including the current major vendors. This ecosystem has proven both wildly successful for developers and widely popular among the end users. Such success probably surprised even Apple, which since its inception had been a closed proprietary operating system – the exact antithesis of the IBM-PC system with an open OS and open arms to any and all programs written to it. Thus did Apple mature and build on the success of iPhone^® with the introduction of iPad^® and soon other iGadgets.

So we urge the vendor community to consider this approach with an open mind and contemplate the endless possibilities.

Vendor Reaction

In the course of my initial research I talked to a number of MLS vendors. I wanted to get their reaction to such a change in technology – was this a valid pursuit? Would it improve the industry and the participants equally, as I had imagined it would? Would they be interested in being considered as the principal technology vendor for such a project?

Reactions to these questions were wide and varied from mildly amused to wildly supportive. The smaller vendors who were struggling to compete were very interested in learning more. Medium to larger vendors were interested, but not too concerned that this might present a threat to their continuing business. And the one system vendor with great longevity and respect in the industry pooh-poohed the idea saying it would never work.

Having now proposed a radical transformation of the MLS infrastructure, our only task remaining will be to find someone to build it. Now that we have deconstructed the MLS, and proposed a path toward reconstruction and resurrection, we will complete our journey and identify a solution provider in Part 4.

In Part I of this series we looked at the MLS industry from a historical perspective, leading us now to a closer look at the particular problems the institution of MLS is facing.

In general, the problems the MLS faces today, that are not institutional, are strictly technology related and fall into two broad categories:

Lack of innovation

MLS systems are woefully inferior compared to other modern technologies. Compare today’s MLS system, used by licensed practitioners, to parallel systems in the investment community or the insurance industry, even the travel industry (both from the standpoint of those few remaining travel agents, but even more so from the consumer’s vantage point).

The lack of innovation isn’t the fault of the system vendors. They have been constrained by the downward pressure on pricing imposed by MLS operators demanding more and more but willing to pay less and less.

Lack of choice

The closed, single-vendor system in each market has stifled choice by agents, by brokers, even by the MLSs themselves. New main system entrants are effectively blocked from entering the market by the long ramp up times from inception to first contract — often years. Third-party application developers who might otherwise fill in the gaps in main system functionality are frustrated by the lack of standardized data layouts and the daunting task of licensing access from 800+ different system operators with different rules and contract stipulations.

Let’s look at each of those areas from the perspective of the major stakeholders – the brokers, the agents and the vendors.

As a broker

In October 2013, The Realty Alliance organization presented the MLS community with a list of “issues” they felt the MLSs had long ignored or refused to fix. They published their list through Clareity Consulting. Many of their concerns were tied to limitations in MLS technology:

• Inability to unbundle products and services – one package, one price, fits all
• Forcing brokers to pay for development of services that compete with their own
• Selling services in competition with brokers who use similar services as a competitive advantage (leveling the playing field)
• Subsidizing associations by overcharging for MLS services
• Making agents and brokers pay for services they don’t want or need
• Non-standard data feeds from contiguous MLSs
• Lack of back-office integration to MLS functionality
• Not doing enough to stop data piracy

As an agent

A real estate agent has a simple list of requirements: give me the tools to do my job and get out of my way. The unspoken expectation is that the tools provided would actually work together, and therein lies the rub.

According to a recent Inman News survey, a lack of integration among various agent tools was a frequent complaint about real estate technology.

Most agents said that at least some of the tools they use are integrated with each other, but which ones actually worked together varied quite a bit. Some involved the combination of two products, such as the MLS and a CMA tool. Many agents suggested that products leave much to be desired when it comes to integration.

“[There are] too many choices that do not work well together to provide seamless transactions for clients,” one agent said.

Another agent asked for a “true contract-to-close” system, combining lead generation, listing management and transaction management.

“All of our ‘stuff ‘ is either disjointed or takes a full-time, patient tech nerd to use,” an agent said.

Another agent lamented that they “waste so much time entering in [the] same information on different platforms.”

“I’d like all of them to work together. Email with CRM with Zillow with my website with the MLS and DocuSign,” an agent said.

The MLS vendors have no incentive at the moment to fix this problem, to bring together all the pieces of the puzzle onto a single development platform that would allow all of these ‘parts’ to contribute to a custom ‘whole’ as has long been wished for by agents across the spectrum. The holy grail of real estate technology, “front-end of choice” – the ability for data to be completely synchronized in real time, across all of an agent’s or broker’s applications, with each application allowing input, as well as the display of listing data – will never exist as long as there is no R&D money to develop it and no profit incentive to maintain it.

As a vendor

The institutional system of one vendor/one market creates one BIG problem. Under the current system, one vendor is picked for one market. Once selected, the vendor has a three-year (more or less) contract on that market and no other vendor can get in to offer competition for even a portion of the entire system because the databases are incompatible.

The database drives every piece of MLS technology because without the central data repository none of the peripheral applications can operate. But industry technology has been architected from its very inception to be a closed, proprietary database system with specific applications created by the vendor that operate only on that particular database. Nothing is interchangeable between vendors. This makes up somewhat for having been bargained down to or even below the breakeven point for the core MLS service, and creates some other advantages (speed, single point of support) – but they are far fewer than the disadvantages.

No interaction with main database

In the single vendor technology scheme, only applications built by the database provider will interact with that database because of proprietary access methods. Some third-party applications can use MLS data but only by exporting the data to the application’s data server or by downloading the data to the user’s computer. This adds an element of lost control and is a continuing concern for MLSs seeking to manage distribution of their listing data outside of the MLS-IDX-VOW-Broker-Agent-Client pipeline.

Changing systems nightmare

The single vendor system made changing MLS vendors a monumental undertaking requiring months of planning, training, parallel operation, and headaches. Every agent must learn a new system and do so in a fairly short period of time while trying to maintain a continuing book of business. Consumers easily jump from one portal site to another with no training whatsoever, but ask a group of agents to change from one MLS system to another and you have a six-month training program to roll out while simultaneously handling the storm of social media protest on Twitter. It’s insane.

Because the two MLS vendors never use the same database structure or field specifications, data conversion from one to the other has been a nightmare. Somewhere along the process, data is nearly always lost, either because some data is impossible to convert (saved searches in one system don’t work in the other; the contract manager of one doesn’t export data in a form that can be read into the other; or templates for presentations and email campaigns all lost) or because there was just no place in the new database to put the old data. The industry is rife with horror stories of MLS conversions having gone badly. In the end, many ask if it was worth the effort. As much as an MLS may want a new vendor with shinier new baubles and beads, the pain and suffering involved in changing is a huge detriment to progress.

No new core players

The single-vendor system institutionalizes the current vendors and stymies new entrants with innovative ideas and newer, more versatile and competitive products from even trying to break into the business. Because most MLS contracts are multi-year, and vendors often try to renew them before the current term is up, it takes years for a new MLS offering to get into the vendor supply chain to even be considered. And in this business, no one wants to go first. So cracking the first contract is a huge hurdle. Finally, unless the new vendor is self-funded, they will find it hard to attract venture capital because the timeline from first investment to first contract and then to first profit can be five years or more, an eternity in the high-tech world.

Besides, in that amount of time the new product would probably be obsolete anyway.

No new applications developed

This structure also blocks independent application providers from entering. App developers quickly find there are just too many MLSs with too many data layouts and too much data to download and normalize to make their product work. While third party developers can offer products under a license that usually involves exporting data from the MLS because the primary vendor won’t allow an outside application to query the vendor database directly, no matter how much more efficient that might be. (That, of course, introduces other problems of distribution control and copyright management, but we won’t get into that right now.)

And let’s face it. Real time access through the RETS interface just doesn’t work fast enough to satisfy the ‘get it now’ mentality of most agents.

No profits

MLS vendors have not fared well in the current system either. Competition is fierce and MLSs are notorious for pitting vendor against vendor in a bidding war at contract renewal time to get the absolute lowest possible user fees, in many cases a price that produces no profit for the MLS vendor but may indeed be a loss leader for other portions of their business (e.g. data collection, aggregation, analysis, and derivative products). With vendor profit margins cut to the bone, one wonders why some of them decide to stay in this business. Yet with minimal margins, MLSs are quick to complain that those same vendors do not do enough R&D work to stay current with changing technologies.

Without profit the MLS vendors have had no money to invest back into product development, market research, usability studies, or technological innovation. Read any number of stories in any of the trade press and you’ll hear a constant theme – the MLSs are losing the beauty contest to the portals because of the lack of reinvestment into their core technology.

So the vendors aren’t making any money; there are no innovations coming out of their R&D efforts because there are no R&D efforts; and there are no new players coming on the scene to give them a reason to innovate. THAT is the perfect formula for disruption and disruption is exactly what I see on the horizon.

The Institution of MLS is in trouble.

I base this conclusion on the amount of turmoil that seems to be swirling around the MLS business lately.

The NAR Core Standards Initiative

NAR launched an initiative in the summer of 2014 to consolidate smaller associations into larger ones. The premise was standards of service that all AORs would have to meet to maintain their charter. Many small ones who cannot meet these standards by mid-2015 would risk being disenfranchised.

The small associations are now figuring this out and looking for alternatives to continue their existence. I’ve been getting calls from very small associations who think they can meet the core standards requirements simply by creating a regional MLS and letting the MLS carry the load of providing the services they need to comply. Such moves are not solving the core problem that core standards were meant to fix – small associations that can’t provide equitable Realtor benefits to all. They’re merely putting a Band-Aid® on the wound when it needs sutures. In the process of trying to improve the industry, the core standards are having the unintended effect of creating issues on the MLS side without fixing the main problem of small AORs.

I think it is highly probable that we will see a flurry of activity in May and June in small associations scrambling to meet the standards, and when they fail to do so, the AOR and its association MLS will shut down rather abruptly (unless, of course, some sort of extension or amnesty is offered).

MLS usage is dropping – pocket listings:

We started to see this phenomenon grow a couple of years ago and now it’s becoming more pervasive to the point where it’s been institutionalized in some MLS systems and ostracized in others.

Some MLSs felt the need to reinforce their relevance in the sales process so they embraced “Coming Soon” listings to offer subscribers a way to load their pocket listings and expose them to the other participants. Other MLSs created rules and penalties for such pocket business practices. This did little more than make some brokers angry because the MLS was now interfering in their business affairs.

The central issue of the Pocket Listing phenomenon is the desire, actually the demand, by agents and brokers to control where, when, and how their listings are not just advertised, but marketed and managed. The home seller hired the brokerage to facilitate the sale of the property as quickly as possible and at the highest possible price. They do not feel compelled to obey arbitrary rules set by a listing system that interfere with their professional opinion about how best to accomplish that sale. They feel if holding the listing off the market in order to expose it privately to a subset of buyers that, in their professional opinion, have a better chance of making an offer at an acceptable price before going to the general market, then they should be able to make that call.

Likewise, brokers take that requirement to a higher level, looking not just at individual listings but also at entire listing inventories. Should they put all their listings into the MLS if the MLS is not working in their best interests? Or should they collect their listings in their own private network and feed them to the MLS only when the MLS is ready to receive them – read: when the MLS will do business their way?

Thus the concept of Project Upstream was born. At its core, Upstream is a more organized, more widespread, more grown-up version of pocket listings.

It was conceived by brokers who were angry. And they’re not just angry about listings, they’re angry with a lot of stuff.

The Realty Alliance

We can all remember where we were in October 2013 when Craig Cheatham, CEO of The Realty Alliance (TRA) spoke to the CMLS convention in Boise about the growing unrest within the brokerage community. He detailed disillusionment brokers had with the lack of responsiveness from the MLS community in addressing their needs. He highlighted (and later published the list of grievances TRA had accumulated over the years. He gave the MLS community a deadline – “You have 10 days.” – to fix the problem or he/they would.

He disclosed Project Upstream, where the brokerages would combine their resources into a national aggregation that would/could displace the MLSs by reversing the flow of listing data. The members would share information with each other first, before sending it to the MLS. It is the “office listing” or “pocket listing” taken to a new level by having large companies involved and cooperating in the effort.

Listings would start on Upstream and then be distributed only to MLSs who complied with the brokers’ demands for more voice in governance. Upstream would also control the flow of listings to the portals (Zillow, Trulia – even <gasp!> Realtor.com) by distributing listings only to those that met their demands for display and lead routing.

It’s not just the TRA Brokers.

TRA is an alliance of non-franchise mega-brokers (although its complexion is changing with the expansion of the Berkshire Hathaway HomeServices brand). But Upstream is supported by not only the independents but also most of the major franchise brands. For example, at the annual KW Family Reunion, Gary Keller, CEO of Keller Williams threw up this slide in the middle of his keynote:

“What should have happened 10 years ago.” That’s how out of touch the head of the world’s largest real estate brand (by agent count) considers the MLS to be. And any MLS that thinks he’s talking about someone else should take a good look in the mirror.

Project Upstream, the broker/MLS co-owned public portal project, the syndication debate, the recently passed AVM data policy, etc. all point to deficiencies in the MLS system. The statement by Mr. Cheatham that his members no longer regard the portals as the main threat, but regard the MLS as the main threat, is easily understood in the context of the heightened awareness of these issues

The list goes on, but you get the idea. In Part III of this series we will look at how most of these issues can be addressed through a new approach to MLS technology.

Evolution of the MLS

Preface

Some of you may have noticed that this blog has been remarkably silent for the past eight months or so. That is not for lack of wanting to comment on happenings in our industry. I have had many thoughts and started numerous posts and then for reasons, which will become clear shortly, I set them aside unfinished or unposted.

For the past 18 months I have been busy working on a project.  The subject of the project is not new — the evolution of MLS technology. The assumptions and proposals of the project are not novel — many were proposed years ago by others. The result of the project is not radical — an extension of current technology into a much broader application.  Yet the project has the potential to revolutionize the MLS industry.

And the concept will be controversial. The stakeholders in all quarters will have opinions about the wisdom of such an approach. And the sayers of “Nay” might as well start sharpening their pitchforks now because I’m sure they will want to shake them at me in the near future.

But given a fair reading and an open minded debate, I think you will find the concepts I am going to relate here to be disturbing to some, interesting to most, and important to all.

My report will be in four parts.
First, a history of how this project developed and the research I undertook.
Part 2 will be a definition and deep dive analysis of the problem the project was designed to address.
In Part 3, I will propose a solution and explain why it makes sense.
And finally, in Part 4, I will look at the pros and cons of the proposed solution and, identify the people I think can provide that solution and look at the potential for the future benefits for all parties involved.

Let’s dive in.

Part I – The Historical Perspective – Background and research.

Summary of the state of the MLS industry

The real estate industry is in a state of flux. Competing groups from all quarters are converging in a conflict that will change the landscape of the infrastructure and it is expected to happen within a relatively short period of time.

At the heart of the conflict is the MLS, as it becomes a focal point for the serious discussions prompted by changes in technology and consumer behavior and a continuing effort by the trade associations that own the majority of MLSs to reestablish their relevance to their membership.

The MLS is being pushed and pulled in many directions, used as a pawn in the chess game between the Realtor associations and the brokerages who formed them and whom they were created to serve. The calls for reform in the practices of the MLSs to make them more responsive to brokerage needs are competing with calls for consolidation and merger to strengthen the existing MLS organizations by increasing their size and influence.

Such conflict is not new. But adding to the growing enmity among these players is the increasing influence of the consumer, particularly the self-service oriented Millennials and Gen-X-Y’s abetted in their pursuits by the growth and influence of the national real estate portals. Zillow in particular has been on a crusade to become the top -of -mind brand when a consumer thinks of anything related to the home buying or selling process, or real estate in general. And they’re succeeding, thanks to a world class marketing team and a seemingly limitless supply of advertising dollars.

I experienced first-hand the animus many MLS CEOs held toward Zillow during the 18 months I called on them as an employee of Zillow, seeking direct listing data feeds. The demands were many and the inclination to negotiate was practically nonexistent. But I consistently felt that the hostility was directed not at me personally or at Zillow as a company, but at an undefined, nebulous “threat” that the MLSs perceived coming from the portals in general. In the minds of some CEOs, the portals were threatening the MLS’s raison d’être by becoming the de facto marketing platform of choice for both agents and consumers.

Into this contentious environment, one MLS CEO made an unusual phone call in early spring of 2013. He said his MLS was about to begin the dreaded vendor selection process and he wondered if Zillow would be interested in being considered as the system provider for their MLS. He was working from the common perception that “Zillow is just one field away from being a national MLS.” Would Zillow like to add that field?

“No!” The answer was swift and resolute. Zillow had no interest in getting into the MLS business or even the technology business on behalf of an MLS. They held to their mantra they were a media company, not a broker, not an MLS, not a title company, nothing but an advertising vehicle for brokers and agents (and others) to reach a potential audience through pay per impression display ads.

That should have been the end of it. But ten days later I received a second call from another association executive asking the same question. I had no reason to believe these two people had talked to each other or compared notes. And I gave the same emphatic negative reply when asked the second time.

That spring, at the NAR mid-year conference in May 2013, I had lunch with both executives and talked in general about their frustrations with the current vendor options and their ideas for a different kind of MLS system. Their comments stayed with me even after I left Zillow some months later.

After Zillow, this newly minted industry consultant was invited to facilitate various strategic planning discussions for MLSs large and small. Many, I found, were wrestling with the same questions, looking for answers that would define purpose and mission for an industry seemingly at sea.

The MLSs were feeling pressure from many quarters.

• Big brokers, including the major franchises, had launched a broadside against MLSs in general when in the fall of 2013 The Realty Alliance presented their demands for immediate industry reform — “or else.”
• Agents and brokers expressed more frustration with the lack of innovation in MLS technology and pointed to the upstart portals as examples of what an MLS should look like. MLSs were seemingly talking to the wall if they asked for major improvements in their MLS system from their vendor. The MLSs had dug their own deep hole by playing vendors off against each other during contract renewal time, thus demanding and receiving the lowest possible price for service, sometimes near the break-even point for vendors. Without profits, vendors were at a loss how to finance innovation, research, and new development, while the MLSs who had selected them felt the backlash from their subscribers who demanded more.
• Agents are irritated with the lack of professionalism in the industry and the influx of thousands of new practitioners who became licensed during the bubble and who were now just hanging on. They started pocketing listings, marketing them through peer networks on Facebook rather than submitting them to the MLS. Major chunks of inventory in some parts of the country were being sold off-MLS.
• NAR through MLS rules adoptions and subsequent reversals of such rules were making it more difficult for MLSs to manage the playing field at the local level. The “Core Standards” initiative posed a threat to the charters of smaller associations. While many members would not miss their tiny association, agents would die without their local MLS that could wither with the demise of the association.

As I worked with the MLSs to dissect each of these problem areas, a couple of major trends showed through: most (not all by a long shot) of these problems resulted from either a lack of innovative technology to be able to solve the problem, or if such a solution were present and available then a lack of adoption of that solution by the MLS.

In Part II of this series, we will dig deeper into the challenges of running an MLS in today’s Internet age using systems developed during the first (sic) Clinton administration.