Just an update I can see the data going into the bigtable with the updated values just fine. Not sure if this helps

The issue was that the index files were not being updated with the new data for the page in question. The error logs showed that an error was occuring when the index was trying to index a parent 'Referenced' page which did not actually exist.

protected virtual void SetStandardPagePropertiesInIndexItem(PageData page, IndexRequestItem item)
{
   item.Uri = new Url(page.LinkURL).Uri;
   item.Title = page.PageName;
   item.Created = page.Created;
   item.Modified = page.Changed;
   item.Culture = page.LanguageID;
   item.ItemType = item.ItemType = Enum.GetName(typeof(CmsItemType), CmsItemType.Page);
   item.Authors.Add(page.CreatedBy); 
            
   item.ReferenceId = page.PageTypeName;
}

The reason for this was that the CmsSearchHandler was altered to try and index the PageTypeName which was being pushed into the ReferenceID property. Once this was found to be the issue moving the PageTypename data to be stored in the MetaData solved the issues and the results were now being returned correctly.

protected virtual void SetStandardPagePropertiesInIndexItem(PageData page, IndexRequestItem item)
{
   item.Uri = new Url(page.LinkURL).Uri;
   item.Title = page.PageName;
   item.Created = page.Created;
   item.Modified = page.Changed;
   item.Culture = page.LanguageID;
   item.ItemType = item.ItemType = Enum.GetName(typeof(CmsItemType), CmsItemType.Page);
   item.Authors.Add(page.CreatedBy); 
            
   item.Metadata = Internal.MetaDataQuery.GetExpression("PageTypeName", pageData.PageTypeName); 
}

The 'GetExpression' property can be seen below in my followup reply.

Returning results by searching MetaData

We found that although the data was now being indexed correctly you cannot search just the MetaData using the key 'EPISERVER_SEARCH_METADATA' as this returned no results so in order to search the data stored in the MetaData we had to use a default search 'EPISERVER_SEARCH_DEFAULT'. which obviously searches other content not resticted to the MetaData. In order to make the values in the MetaData unique for search purposes the MetaData was tagged in the index and this tagged data was searched.

[[MetaDataKey=MetaDataValue]]

protected string GetExpression
{
   get
   {
       StringBuilder builder = new StringBuilder();
       builder.Append("[[");
       builder.Append(Property);
       builder.Append("=");
       builder.Append(Value);
       builder.Append("]]");
       return builder.ToString();
   }
}

public virtual string GetQueryExpression()
{
   StringBuilder builder = new StringBuilder();
   builder.Append("EPISERVER_SEARCH_DEFAULT");
   builder.Append(":(");
   builder.Append(GetSafeQuotedPhrase(this.QueryExpression));
   builder.Append(")");
   return builder.ToString();
}

This allowed us to uniquely searchfor particular metadata properties.

If anyone can shed any light on using 'EPISERVER_SEARCH_METADATA' that would be very useful as it would provide a more directed search than that of the default which covers other properties and hence also provide some performance improvements.