ListView cells not refreshing when using Observable Collection and .Remove method

I have a ListView which I want to have a delete MenuItem for in the Toolbar on Android. When a user long presses an item in the list, the trash icon appears in the top right. They tap it, confirm the delete, and the item disappears. A for-loop then runs which is meant to renumber the items, from 1 to the number of items in the ListView, in reverse order.

The issue I am seeing is that some items offscreen don't get updated (the labels are wrong or sometimes the values disappear), even though the ObservableCollection (PersonItems) behind is all correct, and my model is implementing INotifyPropertyChanged. I am using the ObservableCollection's Remove method, and I only set the item source and ObservableCollection once. So I don't think there's any binding-getting-replaced issue here.

Tapping the incorrectly displaying items seems to refresh the values properly. So I'm confident the PersonItems data is fine. How can I ensure every item that the ListView is refreshed?

Here is the code for my remove and renumber methods:

        private void RemoveItemFromList(string itemId)
        {
            var removingPerson = PersonItems.FirstOrDefault(s => s.Id == itemId);
            var isTopItem = removingPerson == PersonItems.First();
            PersonItems.Remove(removingPerson);
            if (!isTopItem) // No need to renumber if only the top item is removed
            {
                RenumberItems(); 
            }
        }

        private void RenumberItems()
        {
            var indexNumber = 1;

            var itemCount = PersonItems.Count;
            for (int i = 0; i < itemCount; i++)
            {
                PersonItems[i].Number = itemCount - i;
            }
            RaisePropertyChanged(nameof(PersonItems));

            //Here, I have tried the same approach with a foreach, but thought the issue might lie with modifying while using an enumerator. But no dice.
            //var indexNumber = 1;
            //foreach (var person in PersonItems.Reverse())
            //{
            //    person .Number = indexNumber++;
            //    RaisePropertyChanged(nameof(person));
            //}
            //RaisePropertyChanged(nameof(PersonItems));
        }

Here are the relevant parts in my model:

    public class Person: INotifyPropertyChanged
    {
        private string _name;
        public string Name
        {
            get => _name;
            set
            {
                _name = value;
                OnPropertyChanged();
            }
        }

        private int _number;
        public int Number
        {
            get => _number;
            set
            {
                _number= value;
                OnPropertyChanged();
                OnPropertyChanged(nameof(Name));
            }
        }

        public event PropertyChangedEventHandler PropertyChanged;

        protected void OnPropertyChanged([CallerMemberName] string propertyName = null)
        {
            PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));
        }
    }

Also, I am using Data Templates as I have 2 versions of the app that need to show a different cell depending on the version (I don't use a combination of the 2 types of cell, just one or the other).