Should the Joomla Extensions user rating system be changed?

[sprocket]

OK, not wanting to sound pretentious or anything, but I honestly think that the current user rating system in no way reflects the quality of the extension....

There are extensions that are extremely complex, but extremely well written that have bad scores, because some users clearly can't configure the thing as it says in the manual and goes editing files left/right/center.

There are others labelled as beta which have 4 stars and on a totally fresh install is nothing but an alpha, because it's a poorly rewritten extension to support legacy Joomla upgrades, but not new installs.

There are some which are excellent extensions, but that don't even figure in the extensions directory.

As I see it, there is no real feedback discussion in the ratings - just a one-way client comment. Shouldn't the comments at least be a digest of user/developer feedback???

Your thoughts?

[pe7er]

I agree.

My opinion (posted at some other thread):
It's a pity that the review system at http://extensions.joomla.org/ is not linked to the user profiles at http://forum.joomla.org/
If it was, it would have given an indication about the users background (it's profile and postings at the forum), and would give an overall indication about the expertise of the user and of its comment.

Someone elses opinion in a thread called "Relevance of reviews": http://forum.joomla.org/index.php/topic,80095.0.html

[brian]

i'm undecided about the value of keeping these comments at all. i've recently been evaluating several extensions and the first one I tried for each task was based on reviews. sadly in each case they were the worst of the extensions for that task. each had glaring errors.
this lead me to conclude that the reviews were either NOT for the currently available release or that the reviews were not genuine.
for now I will not be reading any of the reviews and will instead try each option alphabeticaly until I find the ideal one for the task. [at least that way its my fault if I chose the wrong one]

[MMMedia]

I have mixed feelings on this.  I think it is hard, because you may not know on which version the review was based.  The enviroment can change rather rapidly, (or not change at all in the case of an extension that doesn't evolve to include necessary changes that make it work with an upgrade). 

I am not sure how it could be addressed. 

[unixboymd]

Perhaps the review submission form could be reworked so that you MUST include the version number that you're writing the review for (and any other pertinant info).

[sprocket]

Perhaps the review submission form could be reworked so that you MUST include the version number that you're writing the review for (and any other pertinant info).

I would say that this would be a great and useful idea. And when reading the reviews, maybe a pull-down menu: "Read reviews related to this version". May not work though, as there are not a tonne of reviews.

Certainly, the star rating should either reflect the current version or be a moving average of the all-time score, whichever is higher I guess. Then when you click on the extension to read more, it displays both: this version rating and all versions rating.

[absalom]

It should really move into the framework of a social networking app instead of just 'user-submitted' reviews.

That way you can stop people gaming the system..

[ekaln]

Whether or not this is anyone's fault, arguments about the review system mean that a number of major extensions are missing from the site.

Whatever the cause, that can't be good for Joomla and a solution would be great.

[unixboymd]

What do you mean by "a number of major extensions are missing"?

Whether or not this is anyone's fault, arguments about the review system mean that a number of major extensions are missing from the site.

Whatever the cause, that can't be good for Joomla and a solution would be great.

[ekaln]

I'll PM you with a list.

The point of the post was just to say that way the reviews are done now can cause unhelpful trouble/arguments and it might be better to change the procedures.

[MMMedia]

It isn't up the the Extension Site Team to add extensions to the extension site.  It is up to the developers to submit their extension to the site.  That may be one of the reason why you think some major extensions are missing.  Contact the developers of the extension to find out if they have submitted their extension for inclusion.

Also, be aware that any extension that is deemed to be a security risk or is no longer being actively developed or maintained by the developer, will not be listed on the extension site.

What do you mean by unhelpful trouble/arguments?

[ekaln]

Pm sent.

What do you mean by unhelpful trouble/arguments?

  What I meant was...

Some developers have been annoyed at the handling of extension reviews in the past, lets that not bring those issues up again, but try and find a better way to handle reviews.

[MMMedia]

I don't think anyone likes getting negative reviews.  There actually isn't a way around that.

[unixboymd]

I can agree with that... somewhat. No one likes getting a negative review. But there is a big difference between an objective constructive critisism and an outright bashing. Not to mention the many many reviews that I've seen that are asking for support (in the reviews area).

But you can't force people to leave a "good" negative review instead of bashing.

[unixboymd]

I think the reviews area is good, but there is always room for improvment. Perhaps modifying the way the approval process is done along with better / more clear instructions on what is / isn't an acceptable review and an improved review form.

[brad]

Can I assure you all, the JED editors are working on and addressing this issue...

Now, lets move this to the Extensions Forum

[unixboymd]

Can you assure us Brad? Sure you can I'm all up for some assurance.  j/k (yes I'm slowly going crazy today... anyone wanna come with me?)

Can I assure you all, the JED editors are working on and addressing this issue...

Now, lets move this to the Extensions Forum

[brad]

1.5 Beta is so close, I'm going crazy as well... now, back to the the topic at hand.

[absalom]

I don't think anyone likes getting negative reviews.  There actually isn't a way around that.

There is.

You make it into a social networking app where the forum id is displayed for the person who does the review (which likewise will gives it weighting). That way, people will think twice about how they do negative reviews (the risk is that the system can be gamed in the other direction - however this can be resolved by removing any potential status / weighting from the 'official' teams and moderators)

Cameron Adams did a similar thing with wd06 where the social networking app there allowed other people to tag you, and the best way to manage tags (or any other form of UGC) is to maintain a paper trail as to where it comes from.

http://www.themaninblue.com/writing/per ... 006/09/22/

By taking the power away from the JED admins and placing the responsibility at all times with the end user through social networking/karma, it provides a feedback loop in and on itself.

[unixboymd]

Ok I just had an idea about this (after reading another thread in this category)...

Would it be possible to link a users forum "star rating" to any review that they leave? I for one would beleive a review from a person if they had a high star rating as opposed to someone who had a low or no star rating.

[sprocket]

Ok I just had an idea about this (after reading another thread in this category)...

Would it be possible to link a users forum "star rating" to any review that they leave? I for one would beleive a review from a person if they had a high star rating as opposed to someone who had a low or no star rating.

Well, that wouldn't entice 'lil 'ole me to bother leaving a review 

I think the problem comes with negative reviews. A forum post count doesn't necessarily mean you can write a good review - just means you can write... alot!

Weighting negative comments is a bad idea as well, since that could wipe out a lot of valid positive reviews, and would be open to abuse. I like the tagging idea. Or even flagging.

[absalom]

Ok I just had an idea about this (after reading another thread in this category)...

Would it be possible to link a users forum "star rating" to any review that they leave? I for one would beleive a review from a person if they had a high star rating as opposed to someone who had a low or no star rating.




Well, that wouldn't entice 'lil 'ole me to bother leaving a review 

I think the problem comes with negative reviews. A forum post count doesn't necessarily mean you can write a good review - just means you can write... alot!

Weighting negative comments is a bad idea as well, since that could wipe out a lot of valid positive reviews, and would be open to abuse. I like the tagging idea. Or even flagging.




Tagcloud it and deploy it based on XFN relationships to your forum profile, then link that forum profile to other instances of you across the Web (if you post anywhere else, does that provide a better measurement to whether or not you've written a good post ? maybe)

This then takes care of:

• Accountability inside the community
  
• Validity outside the community
• Credibility inside the community - those who have the most posts here may not be the most valuable
• It then moves away from an arbitary figure judgement - star, post number, role (which usually forms most dating / social networking scripts / social moderation behaviour) to a representative mean. It does stop "positive" or "negative" gaming of the system by moderators in this respect to inflate or deflate credibility of a project listed