I first heard about James Achilles Kirkpatrick on a visit to India in February 1997.

I had just finished a book on the Middle East, four years’ work, and was burnt out. I went to Hyderabad to get away from my desk, to relax and recover. It was spring, and it was while wandering around the city that I  stumbled upon the old British Residency, now the Osmania Women’s College. It was a vast Palladian villa, in plan not unlike its contemporary, the White House in Washington, and it lay in a garden just over the River Musi from the old city. The Residency was now in a bad way. Inside, I found plaster falling in chunks from the ceiling of the old ballroom. As the central block of the house was deemed too dangerous for the students, most of the classes now took place in the former elephant stables at the back.

The complex, I was told, was built by Colonel James Achilles Kirkpatrick, the British Resident at the court of Hyderabad between 1797 and 1805. Kirkpatrick had gone out to India full of ambition, intent on making his name in the subjection of a nation; but instead it was he who was conquered, not by an army but by a Hyderabadi noblewoman called Khair un-Nissa. I was told how in 1800, after falling in love with Khair, Kirkpatrick not only married her, according to Muslim law, and adopted Mughal clothes and ways of living, but had actually converted to Islam and had became a double agent working against the East India Company and for the Hyderabadis. I thought it was the most fascinating story, and by the time I left the garden I was captivated, and wanted to know more. The whole tale simply seemed so different from what one expected of the British in India, and I spent the rest of my time in Hyderabad pursuing anyone who could tell me more. Little did I know that it was to be the start of an obsession that would completely take over my life for the next five years.

Beneath the familiar story of the British conquest and rule of the subcontinent, I found that there lay a far more intriguing and still largely unwritten story – about the Indian conquest of the British imagination. During the eighteenth and early nineteenth century it was clear that it was almost as common for westerners to take on the customs, and even the religions, of India, as the reverse. These White Mughals had responded to their travels in India by slowly shedding their Britishness like an unwanted skin, adopting Indian dress, studying Indian philosophy, taking harems and adopting the ways of the Mughal governing class they slowly came to replace – what Salman Rushdie, talking of modern multiculturalism, has called chutnification. Moreover, the White Mughals were far from an insignificant minority. The wills of the period show that in the 1780’s over one third of the British men in India were leaving all their possessions to one or more Indian wives.

Back in London, I searched around for more about Kirkpatrick. My first real break came when I found that Kirkpatrick’s correspondence with his brother William, had recently been bought by the India Office Library. At first, however, many of the letters seemed disappointingly mundane – gossip about court politics or the occasional plea for a crate of Madeira wine – and initially I found maddeningly few references to Kirkpatrick’s love affairs. Moreover, much of the more interesting material was in cipher. It was only after several weeks of reading that I finally came to the files that contained the Khair un-Nissa letters, and some of these, it turned out, were not encoded. One day, as I opened yet another India Office cardboard folder, my eyes fell on the following paragraph written in a small, firm, sloping hand:

"The interview when I had a full and close survey of her lovely person lasted during the greatest part of the night. At this meeting I attempted to argue the romantic young creature out of a passion which I could not, I confess, help feeling myself something more than pity for.  She declared to me again and again that her affections had been irrevocably fixed on me, that her fate was linked to mine and that she should be content to pass her days with as the humblest of handmaids. Until such time the young lady’s person was inviolate but was it human nature to remain proof against another such fiery trial? I think you cannot but allow that I must have been something more or less than a man to have held out any longer..."

Soon after this I found some pages of cipher which had been overwritten with a ‘translation’, and the code turned out to be a simple one-letter/one-number correspondence. Once this was solved, the whole story quickly began to come together.

Hyderabad in 1800 was a frontier town a bit like post war Berlin or Vienna: a city alive with intrigue and conspiracy, where the British and the French were vying with each other for dominance. Soon after Kirkpatrick had managed to surround and disarm the French in Hyderabad, he had gone to a victory party. It was there that he glimpsed Khair un-Nissa for the first time. Despite the fact that she was only fifteen, was in purdah, a Sayyida, and moreover already engaged to a leading Mughal nobleman, the two had fallen in love, and as a contemporary chronicle put it: "When the story of their amours became public, a general sensation took place. The relations of the Begum were naturally very furious and for a time the life of the lovers was in danger, but their passion for one another was not of a character as could be restrained by fear or disappointment. Every obstacle thrown in their way only seemed to make it stronger and stronger …”

As the scandal spread, Khair un-Nissa’s grandfather threatened to go to the central mosque and raise the Muslims of Hyderabad against the British, and Kirkpatrick was ordered by his superiors to stop seeing the girl. He was forced to agree, and everyone believed the affair is over. But what none of the men knew – and what all the women in the harem were all too aware of – was that Khair was now three months pregnant.

Before long Khair’s pregnancy became public and rumours reached the Governor General in Calcutta that James had raped Khair. When Kirkpatrick was charged with this crime, the Hyderabad Prime Minister cut a deal with James: he would testify to James’s innocence, and allow James to marry Khair, but there was a quid pro quo. James had to promise to “strive for the best interests of the your government and will obey all your orders”- in other words to become a Hyderabadi agent.

For four years I beavered away reconstructing the story. Gradually the love story began to take shape. It was like watching a Polaroid develop, as the outlines slowly established themselves and the colour began to fill in the remaining white spaces. On the last day of my final visit to Hyderabad, I spent the afternoon looking for presents in the old city. It was a Sunday, and everything was half-closed; but I had forgotten to buy presents for my family, and with my plane to Delhi due to take off in only five hours’ time, I frantically trailed from shop to shop, looking for someone who could sell me some Bidri metalwork. Eventually a boy offered to take me to a shop where he said I could find a Bidri box. He led me deep into the back of the bazaar, behind the Chowk Masjid. There, down a small alley, lay a shop where he promised I would find ‘booxies booxies’.

The shop did not in fact sell boxes, but books (or ‘booksies’, as my guide had been trying to tell me). Or rather, not so much books as Urdu and Persian manuscripts. These the proprietor, had bought up from private Hyderabadi palace libraries when they were being stripped and bulldozed throughout the sixties and seventies. They now lay stacked from floor to ceiling in a dusty, ill-lit shop the size of a large broom-cupboard. More remarkably still, the man knew exactly what he had. When I told him what I was writing he produced from under a stack a huge, crumbling Persian book, the Kitab Tuhfat al-Alam. The book turned out to be a six-hundred-page autobiography by Khair un-Nissa’s first cousin, written in Hyderabad in the immediate aftermath of the scandal of her marriage to James. Its contents completely transformed my book.

The story gradually emerged of how James had secretly converted to Islam and married Khair. Soon after Khair gave birth to a son, named Sahib Allum (‘Little Lord of the World’), and daughter, Sahib Begum (‘Lady of High Lineage’). To accommodate his new family James began a major building project: to design the vast Palladian villa I had seen as his new official Residency, complete with  British-style park and grazing sheep. Behind it he constructed a Mughal zenana for Khair built in marble with fountains and Mughlai wall paintings, as well as a Mughal garden.  For four years James slipped very happily between these two worlds: by day, he lived his official life with one language and one set of clothes, while in the evening he would get into his kurta pyjamas, and step into the parallel world of his Mughal wife and his Urdu-speaking Muslim family.

This unlikely arcadia finally came to an end after five years when James decided that it would be best to send the children back to Britain to receive an English education and so be able to choose between their two worlds. Despite Khair’s protests, the children were torn from her arms and sent to England, having first had their picture painted by the artist Chinnery  wearing their old Hyderabadi clothes for the last time. On arrival in London they were baptised Christians and given the names William and Kitty Kirkpatrick. James, who now very ill with hepatitis, tried to get to Madras to see them off, but caught a fever and missed them. Soon afterwards, ordered to travel straight onto Calcutta to brief the new Governor General, he became seriously ill and finally died in Calcutta, aged only 43, far from everyone he loved and all who loved him. Khair was left a widow, aged only 19.

By 2001, four years into the research, there still remained important gaps. In particular, the documents gave no hint as to what had happened to Khair after Kirkpatrick’s death. It took another nine months of searching before I stumbled across the heartbreaking answer to that in some papers in the Bodleian Library in Oxford. The tale – which had never been told – bore a striking resemblance to Madame Butterfly.

It emerged that after a year of mourning, Khair had decided to make an epic journey on elephants back, one thousand miles across the length of India, to visit her husband’s grave in Calcutta. Lonely and despairing and far from home, she was eventually seduced by the only man she knew in Calcutta, James’s former assistant, Henry Russell. But Russell was a very different man from James and had refused to marry her. Worse still, when the news of Khair’s seduction by Russell reached Hyderabad, Khair was banished to a scrappy coastal town where she waited in vain for Russell to join her. Russell, however, had other plans, and soon afterwards married a young British heiress in Madras. Khair, broken hearted, wasted away. She was allowed back into Hyderabad to die where she had once been happy: in the zenana of the Residency that James had built for her years earlier.

The final remarkable twist in the story took place a full forty years later. I only came across it a few weeks before I began writing, when family papers belonging to the great-great-great-grandson of Kirkpatrick and Khair un-Nissa turned up a couple of miles from my home in West London. These letters extended the story through to the no less remarkable tale of Khair and James’s daughter, Kitty Kirkpatrick.

After leaving Hyderabad, Kitty had been completely cut off from her maternal relations. One day in May 1841, visiting friends in the Home Counties, she was taken to tea in a stranger’s house, walked in the door and promptly fainted. On the wall was the Chinnery portrait of her painted in Madras when she was four.

It transpired that the house belongs to her mother’s seducer, Henry Russell, who had retired to England with a corruptly aquired fortune and a baronetcy. Kitty began to investigate what her portrait was doing on Russell’s wall and while uncovering the truth about her mother’s end, also discovered that her grandmother was, remarkably, still alive aged 85 in a Hyderabadi harem. The two begin an emotional correspondance: Kitty writing on Basildon Bond from Torquay; the grandmother responding by dictating to a scribe in letters illuminated with gold leaf.  Kitty wrote –

“I often think of you and remember you and my dear mother. I often dream that I am with you in India and that I see you both in the room you used to sit in. No day of my life has ever passed without my thinking of my dear mother. When I dream of my mother I am in such joy to have found her again that I awake, or else am pained in finding that she cannot understand the English I speak. I can well recollect her cries when we left her and I can now see the place where she sat when we parted, and her tearing her long hair – what worlds would I give to possess one lock of that beautiful and much loved hair! How dreadful to think that so many, many years have passed when it would have done my heart such good to think that you loved me & when I longed to write to you & tell you these feelings that I was never able to express, a letter which I was sure would have been detained & now how wonderful it is that after 35 years I am able for the first time to hear that you think of me, and love me. I thank God that he has opened for me a way of making the feelings of my heart known to you.”

Granny responded in Persian, enclosing the lock of Khair un- Nissa’s hair she had kept all that time for Kitty: “Fresh vigour was instilled into my deadened heart and such immeasurable joy was attained by me that it cannot be brought within the compass of being written or recounted. My Child, the Light of my Eyes, the solace of my soul, may God grant you long life!”

The two made plans to meet but tragically Sharaf un-Nissa dies on the eve of the Indian Mutiny- the cataclysm that sweeps away for ever the hybrid world of the White Mughals.

The story of a family where three generations drifted between Christianity and Islam and back again, between suits and salwars, Mughal Hyderabad and Victorian London, seemed to me to raise huge questions: about Britishness and the nature of Empire, about faith, and about personal identity; indeed, about how far all of these mattered, and were fixed and immutable – or how far they were in fact flexible, tractable, negotiable. Yet clearly – and this was what really fascinated me – while the documentation surrounding Kirkpatrick’s story was uniquely well-preserved, giving a window into a world that few realise ever existed, the situation itself was far from unusual.

The Kirkpatricks inhabited a world that was far more hybrid, and with far less clearly defined ethnic, national and religious borders, than we have all been conditioned to expect.  Only 75 years after the death of James Achilles Kirkpatrick, and indeed within the lifetime of his Anglo-Indian, Torquay-Hyderabadi, Islamo-Christian daughter, it was possible for Kipling to write that ‘East is East and West is West and never the twain shall meet’. There is a tendency to laugh at Kipling today; but at a time when respectable academics talk of a Clash of Civilisations, and East and West, Islam and Christianity appear to be engaged in another major confrontation, this unlikely group of expatriates provides a timely reminder that it is indeed very possible – and has always been possible – to reconcile the two worlds. It is only bigotry, prejudice, racism and fear that drive them apart. But they have met and mingled in the past; and they will do so again.