Seven rules for automation: the Ethical Technology Charter

The firm uses models in two of its six divisions. Quantitative Strategies runs pricing, signal and risk aggregation models. Information & Data builds sector datasets and the analytics sold to institutional clients from them. Nothing in either division decides anything about a client without a person. That position is easy to hold at the present size and difficult to hold later, which is why the Board adopted the Charter now rather than after the first system that would have tested it. The Ethics & Technology Council, chaired by the Head of Technology and Ethics, oversees it and reports to the Board.

The first principle is human accountability. A named person answers for every automated decision. In practice this means the model inventory for the group carries an individual name against each entry, not a team and not a division, and that person is accountable for the output whether or not they wrote the code. The second is proportionate automation: automation is permitted only where the cost of the system being wrong is understood and stated. Where that cost has not been quantified, the work stays manual. Three internal proposals have been withdrawn under this principle since drafting began, two of them in client onboarding.

The third principle is provenance of data. Every dataset has a documented origin and a lawful basis recorded before it is used. The provenance register that gives effect to this principle is now live and will be maintained by Information & Data from the new Toronto office. The fourth is restraint in inference: the firm makes no inference about an individual beyond what the mandate requires. A client who asks for portfolio construction has not asked to be profiled, and the difference is written into the analytics specifications rather than left to the judgement of the analyst building the model.

The fifth principle is welfare in biological capital, and it is the one that surprises technology people who read the Charter. Animal and biological welfare standards travel with the capital into every life-science position the firm takes. It sits in a technology charter because the firm measures welfare with data, and data collected without a rule becomes surveillance of a farm or a facility rather than assurance. The principle sets what may be collected, from whom, and what may be inferred from it, on the same terms as any other dataset.

The sixth and seventh principles face the client. Transparency to clients means a client can see which decisions were model-assisted, which requires the reporting system to carry the label rather than the relationship manager to remember. The right to a human decision means any client may require a human review of anything a model touched, with a named person responding. Both principles are being built into client reporting this year, and the Private Wealth division will publish the first labelled statements in the fourth quarter.

The Charter is enforced through two policies rather than through goodwill. The Model Risk Management policy, effective in February, inventories every model, tiers it by the cost of being wrong and fixes its validation cycle. The Artificial Intelligence Governance policy applies the Charter to learning systems and holds a register of approved uses. Vendors are bound through the Third-Party policy: of 34 technology contracts reviewed since October, 11 required amendment, and two vendors declined and were replaced. A principle that cannot be put into a contract is a preference, and the firm has been careful to write seven that can.

Published 2023-03-14 by the Information & Data division. Research is prepared for eligible counterparties and does not constitute advice.