ContactSyncService: Implement related functions

[DaVinci9196]

Includes:

1. After adding/deleting/modifying contact groups and personnel information, the cloud and device are synchronized with each other.
2. After adding/deleting/modifying contact avatar information, the cloud and device are synchronized with each other.

Note:
Tried to use PeopleApi, but did not have access to the interface.

[Sense101]

@DaVinci9196 so can people api not be made to work with MicroG? This is critical, as since a few weeks ago all requests made to the old apis return:
Received 429 status code: Contacts API has been deprecated. Migrate to People API to retain programmatic access to Google Contacts. See https://developers.google.com/people/contacts-api-migration.

This includes the google contacts sync adapter

[Sense101]

@DaVinci9196 I would like to help to finish implementing this. Please respond.

[DaVinci9196]

@DaVinci9196 I would like to help to finish implementing this. Please respond.

Sorry, I just noticed today that I used OauthToken generated based on GMS and Google Contacts app when verifying whether PeopleApi is available. Unfortunately, errors such as unable to access the API and no contacts permission appeared. You can try to see if there are other ways to solve these errors.

[Sense101]

I am looking into this

[Sense101]

I am still looking into this, as the entire structure of the api has changed this is not as simple as I thought.

[000exploit]

@Sense101 sorry for bothering, but what have you discovered?

[Sense101]

No problem, took a break from this although I will be returning to it. Part of what I was working on is that the format of the .proto files (which define data sent between the client and server) must include specific annotations to link up correctly with the new google api's. This required different dependencies to the ones used right now in MicroG, which was a whole mess to sort out.

Still lots of work to do, but that's part of what I discovered :)

[Sense101]

Just noting I had some incorrect assumptions about this.

[Sense101]

@DaVinci9196 I have been working on this, and have replicated calling the People api using an oauth generated based on GMS. I got the following error messsage back from google:

People API has not been used in project 745476177629 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/people.googleapis.com/overview?project=745476177629 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

Was this the same error that you saw? If not, what error did you see?

I tried generating oauth token based on the old contacts sync adapter id but couldn't get an auth token.

I think this is a crucial feature for microg to have so it should be a priority to fix.

The only next step I see from here is reverse engineering how play services does it, which I have no experience with.
It seems likely that google play services is using some internal API - would you be able to either find out what that is / know how I could do that?

If anyone else is willing to help with this it would also be appreciated.

[DaVinci9196]

@Sense101 Yes, we have encountered this problem since the beginning. This is a configuration behavior of GMS itself and there is no way to bypass or fix it. The current solution was adopted as the last resort to sync contacts.

[Sense101]

@DaVinci9196 so to clarify, you're saying there is no way to now include contacts sync within MicroG?
Why is reverse engineering what play services does not an option?

[DaVinci9196]

@Sense101 At least not yet, maybe Marvin is considering other options.

[ale5000-git]

@DaVinci9196
@Sense101

Before implementing the contact sync part have you tried to implement the backup transport part?

On Android 4.4 (Kitkat) the Google Contact Sync Adapter behave in this way:

• WithOUT Google Backup Transport it use its internal deprecated sync over plain http
• With Google Backup Transport installed it use Google Backup Transport to do the sync

Before Android 6 (Marshmallow) all Google Backup Transports include the service: com.google.android.backup.BackupTransportService
Starting from Android 6 the service .backup.BackupTransportService was removed from Google Backup Transport and added to Google Play Services.

On Android 7.1.2 (Nougat) Google Contact Sync Adapter say this:

09-12 19:36:24.107  6027  6046 I SyncAdapterProxy: Delagator disabled, using the (deprecated) GData sync adapter
09-12 19:36:24.110  6027  6046 I GoogleHttpClient: GMS http client unavailable, use old client

So if we implement the missing parts in microG Services (possibly service .backup.BackupTransportService) maybe the official Google Contact Sync Adapter may start working again.

[DaVinci9196]

@ale5000-git Compared to implementing another larger function, I prefer the current PR solution :)

[ale5000-git]

The whole file is 0.17 MB, I wouldn't say it is big.
This is the one I'm referring to: https://www.apkmirror.com/apk/google-inc/google-backup-transport/google-backup-transport-5-1-1-2640980-release/google-backup-transport-5-1-1-2640980-android-apk-download/

Also the current PR don't work, or I'm wrong?

[Sense101]

As @ale5000-git said, this PR doesn't work anymore, and there isn't any way to get it working.

However, I don't think relying on Google Contacts sync apk is a good solution because Google clearly stopped producing further versions and there's no way it will work long term.

The idea with backup transport may be a good try though for a short term solution, although I don't like the idea of adding old code

[DaVinci9196]

After verification, this PR still works to obtain contact data. However, additional steps are required. After granting microG permission to read your address book, select your Google account on the Accounts page in System Settings and click Sync Now.

[ildar]

After verification, this PR still works to obtain contact data.

This is awesome! Is there an early build for alpha-testing? thanks.

P.S. I see that CI doesn't leave artifacts, sadly

[Sense101]

Hmm so it seems the shutdown of the old api for contacts sync didn't yet impact when play services is asking for sync?

That's unusual as it is shut down for all other apps
I think it is only a matter of time till they shut it down fully

[Careless]

pardon my resurrection of this PR, but is the GoogleContactSync officially dead as far as microG is concerned?

I'm looking at the 3173 PR and it says "won't fix"... so does this mean the contactsyncadapter will never work again, or there will not be any further development on integrating it into microG?

[ale5000-git]

@mar-v-in
Any plan for this?
I think it's very important for a large portion of users.

[Sense101]

Can confirm that this current solution is functioning right now. If there are plans to integrate this into MicroG more officially, then there are a few changes I can PR for improvement.

For example, we discovered that the current max contacts this will pull from Google on first sync is 1000. I believe repeated calls with pagination may be needed in that case, although I haven't yet looked into how the old API differs from People API regarding that

[xisnamh]

Puedo confirmar que esta solución actual está funcionando en este momento. Si hay planes para integrar esto en MicroG de manera más oficial, entonces hay algunos cambios que puedo promocionar para mejorar.

Por ejemplo, descubrimos que el máximo actual de contactos que esto extraerá de Google en la primera sincronización es 1000. Creo que en ese caso pueden ser necesarias llamadas repetidas con paginación, aunque todavía no he analizado en qué se diferencia la API anterior de la API People al respecto

Please, can you say the steps correctly for work the sync contacts?? thanks so much

[Careless]

Can confirm that this current solution is functioning right now. If there are plans to integrate this into MicroG more officially, then there are a few changes I can PR for improvement.

Either I'm blind, or confused... or both... 🫠

What is the current solution and how do we use it?

[ale5000-git]

What is the current solution and how do we use it?

This is just a PR so the only solution currently is to compile the code yourself.

[floomat]

What is the current solution and how do we use it?

This is just a PR so the only solution currently is to compile the code yourself.

As there are no conflicts left how long does it take to get merged into the next release then? It actually will be a huge benefit for a lot of users.

[Sense101]

Upon further inspection this PR is currently not feature complete:

• If a google account has more than 1000 contacts, only the first 1000 are downloaded to the device
• All contacts synced to the device are not properly saved under "My Contacts", but instead as "other" contacts. This results in being unable to search for any contacts on Google Contacts, Dialer or Messages, amongst other problems. This can be proven by going onto a synced contact page, instead of the edit button on the top right, you will see an "Add contact" button.

There may also be additional issues not yet found, I will be doing further testing.

[xisnamh]

Tras una inspección más detallada, este PR actualmente no está completo:

• Si una cuenta de Google tiene más de 1000 contactos, solo los primeros 1000 se descargan al dispositivo
• Todos los contactos sincronizados con el dispositivo no se guardan correctamente en "Mis contactos", sino como "otros" contactos. Esto da como resultado que no se pueda buscar ningún contacto en Google Contacts, Dialer o Messages, entre otros problemas. Esto se puede demostrar yendo a una página de contacto sincronizada, en lugar del botón editar en la parte superior derecha, verá un botón "Agregar contacto".

También puede haber problemas adicionales que aún no se han encontrado, haré más pruebas.

the general people dont need 1000contacts, you can put a comment when the fuction its added saying "only works with less 1000contacts for sync correct".
not is a fuction necessary for normal people hahah but we need the contacts sync works correctly and not depends of external solutions. thanks for your work ;)

[Sense101]

If any other developer knows whether this is still the primary API google themselves use for GMS Contacts Sync, or could check that, it would be very helpful. Otherwise, google might just shut this down after we implemented it.