Rocket.Chat integration partly works, but exclamation mark remains

mabus 2020-02-04 10:48:01 UTC #1

Hi. I’m new to this forum (and to eGroupware), so first, hello everbody.

I installed egroupware-docker one week ago and configured everything to my needs. I also integrated egroupware-rocketchat (docker) mostly succesful. That means, I get logged on to Rocket.Chat via oauth automatically when opening Rocket.Chat on the sidebar and everything works perfectly on that side.

A problem that still remains is the little exclamation mark at my user icon (top right of the screen, see screen shot). The tooltip says there was a connection issue with Rocket.Chat.

egroupware-excl

Here’s some info:

eGroupware: 19.1.20200130 (docker)
OS: Linux: Ubuntu 18.04.3 LTS
PHP: 7.3.14-1+ubuntu18.04.1+deb.sury.org+1: fpm-fcgi
Rocket.Chat: 2.4.2

Rocket Chat was configured as local server, the whole system runs on the internal network and is not accessible from the outside. I have not yet configured SSL/https and DNS for this system, so everything is addressed via IP (http://192.168…). I followed the instructions on OAUTH integration given within “admin/apps/rocket.chat/Install Information” and also set the ROOT_URL within /etc/egroupware-rocketchat/docker-compose.yml

Thanks.

RalfBecker 2020-02-05 08:12:23 UTC #2

If it’s working in the browser, it usually means it does not work from the EGroupware Container to Rocket.Chat container. Either have a look at the logs:

cd /etc/egroupware-docker
docker-compose logs -f nginx 2>&1 | sed 's/PHP message/\nPHP message/g'

Or try from within the EGroupware container (use the URL as specified in EGw and append “api/info”):

cd /etc/egroupware-docker
docker-compose exec -it egroupware bash
curl -i http://192.168..../rocketchat/api/info

Ralf

mabus 2020-02-05 10:07:23 UTC #3

Ralf,

thanks, your help is very appreciated.

To have a clear log, I rebooted the system, logged in (still exclamation mark), opened rocket-chat via sidebar, and logged out. That’s the output:

/# docker-compose logs -f nginx 2>&1 | sed 's/PHP message/\nPHP message/g'
<...>
egroupware-nginx | 2020/02/05 08:45:03 [error] 6#6: *3 FastCGI sent in stderr: "
PHP message: PHP Warning: session_id(): Cannot change session id when session is active in /usr/share/egroupware/api/src/Session.php on line 547
PHP message: #1 /usr/share/egroupware/api/src/Session.php(547): session_id('obfuscated...')
PHP message: #2 /usr/share/egroupware/api/src/Header/Authenticate.php(109): EGroupware\Api\Session->create('team', 'obfuscated..', 'text', true)
PHP message: #3 [internal function]: EGroupware\Api\Header\Authenticate::autocreate_session_callback(NULL)
PHP message: #4 /usr/share/egroupware/api/src/Egw.php(303): call_user_func_array('EGroupware\\Api\\...', Array)
PHP message: #5 /usr/share/egroupware/api/src/Egw.php(199): EGroupware\Api\Egw->verify_session()
PHP message: #6 /usr/share/egroupware/api/src/Egw.php(64): EGroupware\Api\Egw->setup(Array, true)
PHP message: #7 /usr/share/egroupware/api/src/loader.php(117): EGroupware\Api\Egw->__construct(Array)
PHP message: #8 /var/lib/egroupware/header.inc.php(81): require_once('/usr/share/egro...')
PHP message: #9 /usr/share/egroupware/groupdav.php(36): include('/var/lib/egroup...')
PHP message: #10 {main}
PHP message: # Instance=default, User=, Request=REPORT http://192.168.192.2/egroupware/groupdav.php/calendar/, User-agent=vdirsyncer/0.16.2" while reading response header from upstream, client: 172.20.0.1, server: _, request: "REPORT /egroupware/groupdav.php/calendar/ HTTP/1.0", upstream: "fastcgi://172.20.0.2:9000", host: "192.168.192.2"

older log entries (days ago) show some repeating errors, which may be irrelevant by now. This repeats many times:

PHP message: # Instance=default, User=mabus, Request=POST http://192.168.192.2/egroupware/json.php?menuaction=api.EGroupware\Api\Json\Tail.ajax_chunk, User-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
PHP message: Ein Fehler ist aufgetreten. (EGroupware\Api\Exception\WrongParameter): Not allowed to view 'openid/request.log'!
PHP message: #0 [internal function]: EGroupware\Api\Json\Tail->ajax_chunk('openid/request....', 0)
PHP message: #1 /usr/share/egroupware/api/src/Json/Request.php(192): call_user_func_array(Array, Array)
PHP message: #2 /usr/share/egroupware/api/src/Json/Request.php(104): EGroupware\Api\Json\Request->handleRequest('api.EGroupware\\...', Array)
PHP message: #3 /usr/share/egroupware/json.php(125): EGroupware\Api\Json\Request->parseRequest('api.EGroupware\\...', '{"request":{"pa...')
PHP message: #4 {main}

docker-compose exec -it egroupware bash

I think you meant (and I’m just correcting it for other users who want to try it)

docker exec -it egroupware bash

The output was:

/# sudo docker exec -it egroupware bash
root@1387aa613321:/# curl -i http://192.168.192.2/rocketchat/api/info
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 05 Feb 2020 09:56:47 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
X-XSS-Protection: 1
X-Instance-ID: ..youKnowIliketoObfuscateIDs:-)..
Cache-Control: no-store
Pragma: no-cache
Vary: Accept-Encoding

{"version":"2.4.2","success":true}

I was unable to derive anything Rocket.Chat related from these logs, I still hope you can find a hint on what’s going wrong. Thanks.

RalfBecker 2020-02-05 10:51:53 UTC #4

Hmm, non of the log entries have anything to do with your problem, though to get a log about that problem you should open the log in a terminal and then try to log into EGroupware.

You wrote you loged in through EGroupware into Rocket.Chat and did an OAuth Login for the user you have the problem with. The login with the user showing the exclamation mark, is necessary to we have a token EGroupware itself then uses to talk to Rocket.Chat on behalf of your user and update eg. the status in the right hand sidebar.

You can follow the OAuth request and responses by going to Admin >> Open ID Connect / OAuth server >> Request log, best in an other browser / different EGroupware session.

Ralf

mabus 2020-02-05 15:51:50 UTC #5

Hi Ralf,

I logged in as admin via another browser and logged the login process of another account (btw, all accounts have the exclamation mark):

[2020-02-05 15:39:14] openid.DEBUG: POST /egroupware/openid/endpoint.php/access_token HTTP/1.0
Accept: application/json
User-Agent: Meteor/METEOR@1.8.3
Content-Type: application/x-www-form-urlencoded
Connection: close
X-Forwarded-Proto: http
X-Forwarded-For: 172.19.0.2
X-Real-Ip: 172.19.0.2
Host: 192.168.192.2
Content-Length: 1228
code= OBFUSCATED]&redirect_uri=http%3A%2F%2F192.168.192.2%2Frocketchat%2F_oauth%2Fegroupware&grant_type=authorization_code&state=l[OBFUSCATED]&client_secret=...&client_id=Rocket.Chat
  
[2020-02-05 15:39:14] openid.DEBUG: HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Pragma: no-cache
Cache-Control: no-store   {"id_token":"[OBFUSCATED]","token_type":"Bearer","expires_in":3600,"access_token":"[]"}
  
[2020-02-05 15:39:14] openid.DEBUG: GET /egroupware/openid/endpoint.php/userinfo HTTP/1.0
Authorization: Bearer [OBFUSCATED]
Accept: application/json
User-Agent: Meteor/METEOR@1.8.3
Connection: close
X-Forwarded-Proto: http
X-Forwarded-For: 172.19.0.2
X-Real-Ip: 172.19.0.2
Host: 192.168.192.2
Content-Length: 
Content-Type: 
  
[2020-02-05 15:39:14] openid.DEBUG: HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Pragma: no-cache
Cache-Control: no-store

{"sub":"mabus","email":"my.email@address.de","email_verified":true,"name":"Markus ThatsMe","family_name":"ThatsMe","given_name":"Markus","middle_name":null,"nickname":"","preferred_username":"mabus","profile":"","picture":"https:\/\/www.gravatar.com\/avatar\/90a70more","website":null,"gender":"n\/a","birthdate":null,"zoneinfo":"Europe\/Berlin","locale":"de_DE","updated_at":"2020-01-23","roles":["user","admin"]}

RalfBecker 2020-02-05 18:46:49 UTC #6

That looks ok:

an access-token was issues for an auth-code and
a token was verified via the userinfo endpoint

You could enable more logging:

github.com

EGroupware/rocketchat/blob/master/src/Api/Restapi.php#L38


	const APPNAME = 'rocketchat';


	/**
	 * Api URL
	 */
	const API_URL = 'api/v1/';


	/**
	 * Debug mode switch
	 */
	const DEBUG = false;


	/**
	 * auth session key
	 */
	const AUTH_SESSION = 'auth';


	/**
	 * OAuth scopes used for the access-token
	 */
	const OAUTH_SCOPES = 'openid profile email';

github.com

EGroupware/rocketchat/blob/master/src/Api/Restapi.php#L135


		// authToken and useId can be passed through params to setup the header
		// but do not include them in curl URL.
		unset($_params['X-User-Id'], $_params['X-Auth-Token']);
		$curl = curl_init();
		$curlOpts = [
			CURLOPT_HTTPHEADER => $header,
			CURLOPT_CUSTOMREQUEST => $_method,
			CURLOPT_RETURNTRANSFER => 1,
			CURLOPT_FOLLOWLOCATION => 1,
			CURLOPT_TIMEOUT => 1,
			//CURLOPT_VERBOSE => 1,
			CURLOPT_URL => $full_path,
		];
		if ($_method == "POST")
		{
			$curlOpts[CURLOPT_POSTFIELDS] = json_encode($_params);
		}
		else
		{
			$curlOpts[CURLOPT_URL] .= '?'. http_build_query($_params);
		}

Ralf

mabus 2020-02-07 14:39:29 UTC #7

Ralf, I opened a bash into the running eGroupware container and directly edited (substituted with sed, to be precise) the lines. I set the DEBUG constant to true and uncommented the CURL_VERBOSE assignment.

I did this in both /usr/share/egroupware/rocketchat/src/Api/Restapi.php and /usr/share/egroupware-sources/rocketchat/src/Api/Restapi.php and validated my substitutions were correctly applied.

Afterwards I cleared cache, re-opened the request.log, logged in and out multiple times within another browser and another account. Then I checked both request log and docker logs.

The result was, the logs still look the same. Maybe I did not edit the right files, or should have restarted something after editing. Not knowing the installation details of eGroupware – and being a bit lazy about dissecting the whole setup – I assumed that Apache would serve PHP files right from /usr/share, as I didn’t “find” (the command) Restapi.php anywhere else.

Could you please elaborate a bit further how I can log more details? Thanks!

RalfBecker 2020-02-07 17:19:22 UTC #8

We configured OPCache to not check file timestamps, so you either need to kill -s USR2 1 inside the container, or make your changes in /usr/share/egroupware-sources and docker-compose restart.

Ralf

mabus 2020-02-08 11:34:16 UTC #9

Thanks, Ralf, this helped to investigate further and finally resolve the issue. There were several problems, all of which can be circumvented by doing the Rocket.Chat configuration in a certain manner (below). The root cause is some very strict, implicit handling of the API URL and OAuth service name. Logging the problems revealed a few bugs in the PHP code.

How to correctly configure EGW to circumvent any issues:

In _EGW -> Admin -> Apps -> Rocket.Chat:

Server-URL MUST have a trailing slash AND
Rocket.Chat OAuth Service name MUST be lowercase , no matter if the service name is caps within the Rocket.Chat Admin Interface.

If one of the above is wrong, you get this odd behavior: You can log in to Rocket.Chat via EGW iframe (sidebar), but the status icon will always show the exclamation mark.

Here some pictures to clarify things even more:

Rocket.Chat:
rocketchat-oauth-adm

EGW:

Bugs in the PHP code (@Ralf)

Here some actual bugs apart from the configuration needs above.

Login (and other) errors to RestAPI are not correctly detected. For the login case, this leads to omitted error log entries.

github.com

EGroupware/rocketchat/blob/master/src/Api/Restapi.php#L332


	 */
	private static function _responseHandler ($response)
	{
		if (is_array($response))
		{
			$result = [
				'response' => $response,
				'message' => $response['message'],
				'success' => true
			];
			if ($response['error'])
			{
				$result['success'] = false;
			}
			elseif (isset($response['success']) && $response['success'] == true
					|| isset ($response['status']) && $response['status'] == 'success')
			{
				$result['success'] = true;
			}
		}
		return $result;

$response is not “error” on failed login, but the $response[‘status’] is “error”.
Prove (created with some additional debug code inserted…):

PHP message: EGroupware\Rocketchat\Api\Restapi::login response:Array
egroupware-nginx | (
egroupware-nginx |     [status] => error
egroupware-nginx |     [message] => Unexpected AccessToken service EGroupware Rocket.Chat
egroupware-nginx | )
egroupware-nginx |

Fix:

if ($response['error'] || isset ($response['status']) && $response['status'] == 'error') 
{
    $result['success'] = false;
}
// The code below is obsolete(?) $response['success'] was already set to true.
elseif (isset($response['success']) && $response['success'] == true
  || isset ($response['status']) && $response['status'] == 'success')
{
    $result['success'] = true;
}

By the way, what happens if $response is NOT an array (first if statement). There’s an else branch missing, otherwise a undef’d $result is returned.

I stumbled upon this while analyzing the code, without ever debugging deeper into it:

github.com

EGroupware/rocketchat/blob/master/src/Api/Restapi.php#L110


	 * @param string $_api_path api url
	 * @param type $_method = GET
	 * @param type $_params = []
	 * @return array return an array of response
	 * @throws Exception
	 */
	private function api_call ($_api_path, $_method="GET", $_params=[])
	{
		if ($_api_path[0] == '/')
		{
			$full_path = Api\Framework::getUrl(substr($this->config['server_url'], -1) == '/' ?
				substr($this->config['server_url'], 0,-1) : $this->config['server_url']).$_api_path;
		}
		else
		{
			$full_path = $this->data['api_path'].$_api_path;
		}
		$header = [
			"X-Auth-Token: ".($_params['X-Auth-Token'] ? $_params['X-Auth-Token'] : $this->authToken),
			"X-User-Id: ".($_params['X-User-Id'] ? $_params['X-User-Id'] : $this->userId),
			"Content-Type: application/json"

The ‘.’ concatenation operator has precedence over the conditional ?: operator. See https://www.php.net/manual/de/language.operators.precedence.php - this leads to a wrong request URL in the left case. The whole two lines could be replaced by easy-to-read (but not tested)

 $full_path = Api\Framework::getUrl( rtrim($this->config['server_url'],'/') ) . $_api_path;

Code changes to avoid misconfiguations:

Two simple code changes would avoid any misconfiguration by trailing slashes and uppercase letters…

github.com

EGroupware/rocketchat/blob/master/src/Api/Restapi.php#L78


	/**
	 * Constructor
	 * @param array $_data
	 * @throws Exception\LoginFailure
	 */
	public function __construct($_data = [])
	{
		$this->config = Config::read('rocketchat');
		$this->data = array_merge([
			'api_path' => !empty($this->config['server_url']) ?
				Api\Framework::getUrl($this->config['server_url']).self::API_URL : null,
			'user' => \EGroupware\Status\Hooks::getUserName(),
			'authentication' => $this->config['authentication'],
			'oauth_client_id' => $this->config['oauth_client_id'],
			'oauth_service_name' => $this->config['oauth_service_name'],
		], $_data);


		if (($auth = Cache::getSession(Rocketchat\Hooks::APPNAME, self::AUTH_SESSION)))
		{
			$this->userId = $auth['userId'];
			$this->authToken = $auth['authToken'];

replaced by

Api\Framework::getUrl(rtrim($this->config['server_url']),'/') .'/'. self::API_URL : null,

would always yield a correct $full_path (tested).

github.com

EGroupware/rocketchat/blob/master/src/Api/Restapi.php#L82


	 */
	public function __construct($_data = [])
	{
		$this->config = Config::read('rocketchat');
		$this->data = array_merge([
			'api_path' => !empty($this->config['server_url']) ?
				Api\Framework::getUrl($this->config['server_url']).self::API_URL : null,
			'user' => \EGroupware\Status\Hooks::getUserName(),
			'authentication' => $this->config['authentication'],
			'oauth_client_id' => $this->config['oauth_client_id'],
			'oauth_service_name' => $this->config['oauth_service_name'],
		], $_data);


		if (($auth = Cache::getSession(Rocketchat\Hooks::APPNAME, self::AUTH_SESSION)))
		{
			$this->userId = $auth['userId'];
			$this->authToken = $auth['authToken'];
			if (!$this->me())
			{
				// force a new login with next API call
				Cache::unsetSession(self::APPNAME, 'logged_in');

replaced by

'oauth_service_name' => strtolower($this->config['oauth_service_name']),

would always lowercase the service name.

Rocketchat integration