Subject: bug#16555: closed (Re: bug#16555: 24.3.50; Company and CAPF: dealing with completion values containing extra text)

From: [email-masked] (GNU bug Tracking System)
Date: Tue, 28 Jan 2014 22:52:02 +0000

Your bug report

#16555: 24.3.50; Company and CAPF: dealing with completion values containing extra text

which was filed against the emacs package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
16555: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16555
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#16555: 24.3.50; Company and CAPF: dealing with completion values containing extra text
From: Dmitry Gutov <[email-masked]>
To: Stefan Monnier <[email-masked]>
Cc: [email-masked]
Date: Wed, 29 Jan 2014 00:51:17 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

On 29.01.2014 00:04, Stefan Monnier wrote:
> CAPF needs to improve the annotation
> support, indeed.  I don't think the current annotation-function is
> sufficient, since there are different kinds of annotations.  E.g. adding
> "<f>" is not the same as adding "(int x, float y, Vector<String>)" for
> simple reasons of screen real estate, so in some UIs you'd want to
> display both, while in others you'd only want the short one.

I'm not sure differentiating between them would be beneficial. We 
already have "full document" annotation (company-doc-buffer), "one-line" 
annotation (company-docsig), and just "annotation" itself. If we're 
going to differentiate between different kinds of short annotations, 
this will make 4 different functions a backend would need to define to 
describe a candidate with words.

FWIW, "(int x, float y, Vector<String>)" looks short enough to me. In 
Ruby, it often looks like "(table_name, column_name, [options])", which 
isn't too long and still allows completion-at-point display candidates 
in two columns.

>> We could consider it a bug, though (that annotation-function is called with
>> different objects).
>
> Not really: the "objects" you're talking about are returned by the
> completion-table, i.e. they cover at most one "field" (in the
> completion-boundaries sense), whereas completion-all-completions returns
> strings that can span several fields, so clearly they can't always be
> `eq' to something returned by the completion-table.

I see. Then I'm out of ideas here, and using text properties, as 
non-obvious that is, indeed remains the best option.

> The idea is rather to let the backend provide more kinds of annotations
> and let the UI choose which one to use.  E.g. icomplete-mode doesn't
> want any annotation at all, because its screen real-estate is
> very limited.  So completion-all-completions can't blindly add annotations.

Thanks for pointing that out.

>> Come to think of it, company-backends should be able to use a hash-table
>> with `eq' test maybe already, or if I massage the code a little. The major
>> question for me was about uniqueness, and looks like, yes, doing
>> delete-consecutive-dups' after fetching all annotations should be fast
>> enough (and this approach even has some space for optimization). So that
>> leaves a problem with CAPF.
>
> That sounded like "thinking out loud for myself".  I don't know what you
> wanted to say nor how that relates to CAPF.

It was. Sorry if it's out of place.
I filed this bug for discussing a new feature in both Company and CAPF, 
and that was me summing up the (one-sided) discussion of it on the 
Company side. So, closing.

> FWIW,
> minibuffer-completion-help uses `sort' on the "annotated completions"
> and then display-completion-list uses delete-consecutive-dups (tho hand
> written into the loop).

Yes, delete-consecutive-dups in Company is also inlined (in 
`company-calculate-candidates'), we couldn't use the function itself 
anyway, cause it's 24.4-only.

>> It could be an either/or specification: if annotation-function is defined,
>> use it, otherwise, look up the `annotation' property.
>
> I think that in most cases the annotation function will want to do some
> work rather than just return the content of the annotation (as in the
> sample code in my previous message).

Hm, yes, indeed.

Subject: 24.3.50; Company and CAPF: dealing with completion values containing extra text
From: Dmitry Gutov <[email-masked]>
To: [email-masked]
Date: Sun, 26 Jan 2014 06:11:23 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

When dealing with code completion for programming languages, it's usual
that candidates have associated text - most often, as function names,
they include the argument list.

Related issues:

* When inserting the "common part", we must ignore the extra text.

* The argument list is useful, for example when the candidate is
  inserted in the buffer. After it was explicitly selected, we also
  insert the arguments list, but replace each argument with self-erasing
  field when you type over it. We do that in company-clang and
  company-eclim (and could also do in company-senamtic, were someone to
  request that).

* C-like languages usually provide method overloading, and then the
  argument list is a part of a method's identity. Any metadata is
  associated with the tuple "method name" + arguments, so we want to
  pass the method name with arguments to functions that retrieve
  documentation, location, etc.

* If the completion table values only contained method names,
  `delete-duplicates' would remove all but one of the methods with the
  same name. But we do want to remove duplicates.

At the moment the following approach emerged (for examples, again, see
company-clang and company-eclim): the completion values include the
argument lists (but nothing extra except that), and the respective
backends define an undocumented command: `crop'.

That command is only used in two situations:

1. When `company-complete-common' is called, we insert the common part
among all candidates, but before that we call (backend-function 'crop
"common-part"), to make sure not to insert the paren or anything after
it.

2. When `company-auto-complete' feature is used, typing any character
from `company-auto-complete-chars' insert the currently selected
completion into the buffer. In that case we also only want to insert the
method name, and so backend's `crop' function is called.

When the candidate is inserted normally (by selecting it with M-p or M-n
and then pressing RET), `crop' is not called. The full candidate value
is inserted into the buffer, and then we call `post-completion' backend
command, to allow it to remove the arguments list, or do something more
advanced (see the * #2 above).

This behavior is inconsistent, and `crop' doesn't sounds too good as a
command name.

At the risk of breaking some existing code, and thanks to `crop' stil'
beging undocumented, I'd like to define a `value' command instead that
would return the "cleaned" candidate text, presumably without the
arguments list. It would get called anytime before the candidate text
gets inserted, and to get the above mentioned templatification behavior,
the `post-completion' code will need to explicitly insert the rest of
the candidate text (the full candidate will be passed as the first and
the only argument). Which will be the main difference from the backend's
writer perspective.

I see two problems:

* Including extra text in completion table seems like it won't mesh well
  with the completion-at-point-functions interface, and specifically
  with the completion-at-point as the frontend. How would one write a
  CAPF function for clang or eclim with argument lists in mind?

* `company-update-candidates' currently calls `company--safe-candidate'
  on the "common part", on the assumption that the `crop' command will
  return something meaningful for a string that's not itself a
  completion candidate (and currently, they all do, because they look
  for `(' instead of, say, using text properties or a hash table
  lookup). If the command is called `value', it would be more tempting
  for the implementor to only expect it to be called on actual
  candidates. And then it'll return nil or do something unexpected when
  called on something like "foo(".

Thoughts?