nodalpoint.org - Impact factors - Comments

A Hirsch-type index for journals

PedroBeltrao — Tue, 22 Nov 2005 06:47:26 -0500

The last issue of The Scientist has a letter by Tibor Braun, Wolfgang Glänzel and András Schubert on extending the H-index to journals. They compared the top 10 journals ranked by H-index with the ranking by impact factor to highlight some of the differences between the two ratings. Down go the review only journals and up come some higher volume journals line PNAS and JBC.

OK, so is there anything

Dan S — Wed, 07 Sep 2005 05:20:07 -0400

OK, so is there anything wrong with using the level of non-citation within a journal as a measure of journal quality? The idea strikes me as a nice one. It is vulnerable to being fooled by self-citation, as is JIF, so maybe self-citations should be excluded...?

h index and age

PedroBeltrao — Fri, 26 Aug 2005 11:09:39 -0400

Well you could always take age into account when looking at an age dependent criteria like this h index. The person evaluating should be able to get a compilation of people from the same field with the same amount of research time and calculate a distribution of h index.
Anyway, I guess that no one would like to be represented by a number, the same goes for the number of papers weighted by their impact factors. I think that focusing on just one criteria no matter how good it is will tend to corrupt it.

If you can't beat them, join them !!

jaws — Tue, 23 Aug 2005 16:36:32 -0400

I was reading this article a few days ago. The two main strategies described by this author in order to increase the impact factor of any publication are:

# publishing in high-impact journals
# persuading other scientists to refer to your work

There is yet another method, which is to quote all your previous papers in your new publication, even tho' they may be unrelated. This will drive up the citation index of your papers..

You could always judge the importance of a work by other means like the H-index (which are age-ist), but these are not perfect either.

And what of the things journals get upto in order to improve their impact factors ?

Another publication mentions:

An overview and analysis of the Journal Citation Reports' impact factor is provided here. The historical development, calculation of, and alternatives to impact factor are briefly described. Nine general uses of impact factor, including library collection management decisions, journal rankings, journal decision models, and full-text database evaluation, are discussed. Ten benefits, such as its well-established authority, are listed. Finally, more than a dozen criticisms of citation data in general (e.g., self-citations are counted) and impact factor specifically (e.g., problems with the formula for its calculation) are analyzed. The author concludes that impact factor, if used appropriately and in combination with other criteria, is a valid tool that can assist journal collection management decisions in research libraries.

Age bias?

chris — Mon, 22 Aug 2005 00:49:54 -0400

There's something not quite right about the h-index though. It seems to lean towards older, established authors: the number of highly cited papers you have is only interesting when you have a reasonable amount of papers in the first place, and summing over time requires a long presence in the field. So a young(ish) postdoc with, say, 5-10 papers over the past half-decade gets... zip, actually. Am I missing something here?
It seems a good way of judging people in the second half of their careers, but not before that.

You could argue that *very* young investigators (ie newish postdocs) get rated on potential rather than achievement, but that still leaves the next level stuck in limbo...

The h index

PedroBeltrao — Thu, 18 Aug 2005 10:24:12 -0400

From a Nature news I found this paper on the h index. J. E. Hirsch proposes that the scientific performance for a researcher could be measured by this index h that he defines as : "the number of papers with citation number higher or equal to h".
From the news: "n h-index of 50, for example, means someone has written 50 papers that have each had at least 50 citations."

Such a measure would take the journal out of the rating and stop the current "rich get richer" trend for the impact factor. This way the author will focus more on the audience of the journal.

evaluation

Neil — Thu, 18 Aug 2005 09:45:38 -0400

Sounds like the Czech Republic and Australia have a lot in common.

Whilst impact factors irritate me, I do see why we continue to live the lie. Science funding in Australia is abysmal, competition for grants fierce and most evaluation centres around "track record" - number and quality of publications. Don't get me started on how this impacts young, early career scientists who chose Australia because they value quality of life over career development. I guess if you're the head of a large, well-funded lab in the USA, you can afford to submit to a PLoS journal or similar without worrying that it might impact on your career prospects.

That said, I still think we'd do well to submit to appropriate journals rather than perpetuating this myth of impact and status. I mean Nature maybe, but PNAS? :)

Real use for impact factors?

Dan Svozil — Thu, 18 Aug 2005 08:31:03 -0400

I will limit my comments on the basic research in the field of natural sciences, as I am well aware that applied research and human sciences require different approach.

Impact factor, as a measure of the importance of a specific journal article and the journal in which the article appeared, is by also used to gauge the relative importance of individual researchers, as it is a case in our institute. I have always doubted the validity of this approach, and now I witness sort of witch-hunt here. Personally I regard using impact factors to evaluate individual scientists or their scientific programs as a serious lack of human resources mangement capabilities.

On the other hand, it is true here, in Czech Republic, that our science is in a deep shit with at least 60 percent of non-productive scientists, and there is a strong need to evaluate their usefulness. Moreover, I understand that governemntal bodies and their officers need one simple number to asses the importance of the given research (anything more complicated than one number is beyond their perception, at least in our country).

I'd like to hear about your experience about impact factors, the research evaluation, how is it done in your country, is there something else/better than impact factors?

Dan

Back to impact factors then.

Greg — Thu, 18 Aug 2005 05:53:11 -0400

Back to impact factors then. What are the alternatives ? Or should we just ignore impact factor as a non issue ? A search for impact factor on Hubmed shows that it has been a constant source of frustration and annoyance for many many years. We are scientists, why are their no viable alternatives ?

many admins make light work

Neil — Wed, 17 Aug 2005 20:43:33 -0400

Also Nodal has a number of admins, at least one of whom tends to be logged in at any one time watching for crap. It's more difficult for a single admin running one or more sites, probably in their "spare time".

Funny how a thread about impact factors can morph into a discussion about website admin :)

Sad indeed. I have anonymous

Greg — Wed, 17 Aug 2005 05:44:43 -0400

Sad indeed. I have anonymous comments enabled on nodalpoint, which occasionally attracts comment spamers. I think the site is lucky to be a little bit too niche for anyone to target it directly for comment spam.

My argument in favour of anonymous registration is to allow the occasional reader, who dislikes registering, to comment.

Comment captchas (spell ?) are a possible solution to anonymous comment spammers, but I have yet to feel the need to install the Drupal module.

registration

Neil — Tue, 16 Aug 2005 19:36:50 -0400

I don't mind registering for an individual blog or site in the slightest. However, I'm not keen on signing up to something larger (as seemed to be the case with twoday.net) when all I want to do is post an occasional comment on that blog.

Unregistered comments

FiReaNG3L — Tue, 16 Aug 2005 19:00:11 -0400

Anonymous, unregistered comments are a nightmare for any website. The more popular the site, the worse it gets. Back when my site (Biology News Net) was receiving only 50 hits a day, I had such a system. And I was deleting spam (of the worse kind) twice a day. Always from the same source, but from different IPs. You block a word, they alter it by a letter. A nightmare. And with more traffic, it would have gotten out of control, quick.

Only Slashdot have success with an open model, and they have hordes of mods there. Almost every website and forum that I know of opted for a registration model; sure, comments volume goes way down, but really interested people take the 15 seconds it takes to register. In a perfect world, registration to post comments on blogs wouldn't exist, but in the real world, 0.001% who want to post spam no one reads ruins it for everyone. Sad isn't it?

Hmrg.

Spitshine — Tue, 16 Aug 2005 11:41:16 -0400

Hmrg. Thought that twoday.net (who provide a nice blog hosting service) would allow anonymous comments by now. Everyone is complaining. Looks like the WordPress installation on this box has to hit the web one fine day...

Impact factors

Neil — Tue, 16 Aug 2005 02:01:17 -0400

Spitshine over at A Bioinformatics Blog muses over the importance (or otherwise) of impact factor. I was going to comment over there but it involved some registration with which I wasn't too comfortable (sorry mate). I agree with his post - impact factor has always struck me as one of the great lies of science and one that I don't want to live. I'd even say that how someone feels about impact factor is one of the criteria that I use to determine what kind of scientist they are - the stuffy, traditional unimaginative kind or the enlightened and serene type that inhabit Nodal :). Unfortunately, many people that I know are happy to live this lie. Whenever I'm writing up a paper, there is always discussion about where to send it and the impact factor. It's an entrenched attitude that won't change until more people are willing to just "put it out there", in an appropriate forum, as opposed to worrying about status.