12.24
#...
·
·
3
likes
|
---|
I am one of the submitters, the first line in IOTD process. I spent two years, mostly as a submitter. So I speak from the perspective of someone that has given his time to this community and tried to influence the process in many ways (and failed). I will tell you that it is easy for a great image to be missed - through no fault of the submitter or reviewer - but through many factors outside of the control of the submitters and reviewers as well as the imager. Some of these include: the much larger number of good images, the accident of when a particular image was submitted, the inherent advantage that some imagers have through access to dark and clear skies either through accident or through investment. The current situation is akin to combining men's and women's teams in sports or eliminating weight categories in wrestling - and then blaming the referees when women or those with lower body weight don't win. Or tell them that participation is voluntary after all, and if they didn't participate, it would make things easier for the referees. Or if they are in it to win badges or awards, they should rethink why they are in sports at all. I'm using here arguments that people have made in this thread. What I constantly find in any discussion of changes to the IOTD as it relates to having categories is very high resistance, typically from a very small number of people (relative to the total size of Astrobin). The implied and sometimes stated position taken is that such changes will dilute what they see as their own accomplishments and advantages. It is so everywhere - people who benefit from the status quo will always be resistant to change and the most vocal advocates against it. |
11.02
#...
·
|
---|
Salvatore Iovene:Arun H: The stack ranking process I've suggested here and elsewhere would not require much more than a weekend and would drastically increase the value of this process. We got yet another dust M78 today. A running joke amongst myself and others about how busted this process is. I'm sure the image is nice. Stack rank the process from start to finish. Judges included. Make every level of the process blind to all involved in it (fyi, judges get to see who you are today, for no good reason other than YOLO) and follow basic principles of a real evaluation process used by every other system you know. This isn't rocket science. Why this terrible system remains is beyond me. For Sal's credit, he has made very recent changes due to the fact that his Reviewer community went comatose, but the real problems won't get fixed until every level is blind, and no one cherry picks. Bill |
7.90
#...
·
|
---|
I'd rather Salvatore to spend time on important features (such as a personalized heat-map of your own images, hint hint...) rather than that bloody IOTD. Abolish it and replace it with AI generated fakes for all I care. |
11.02
#...
·
·
1
like
|
---|
andrea tasselli: IOTD is a paid feature for customers to submit data to. Don't be obtuse. |
6.68
#...
·
·
2
likes
|
---|
Bill Long - Dark Matters Astrophotography:I am one of the submitters, the first line in IOTD process. It is a bit different than you remember from your previous experience. Submitters log into the system and go through images that other 20 submitters will have in their queue. This allows more submitters to be able to pick each specific picture. The pictures I (and other 20 submitters) will see need 3 picks to be promoted to the next level. I (like other submitters) have 6 slots. Not 3, as in the past and as you remember. These changes were implemented to allow larger spaces in the net for pictures to pass to the next level. |
7.90
#...
·
|
---|
Bill Long - Dark Matters Astrophotography: I very much don't recall this being anywhere near the contract when I signed up to it. |
6.68
#...
·
|
---|
Bill Long - Dark Matters Astrophotography:andrea tasselli: Can you please develop this? Thanks. |
11.02
#...
·
·
1
like
|
---|
Bill Long - Dark Matters Astrophotography:I am one of the submitters, the first line in IOTD process. You pick things out of sea of more than you have on hand. If you do not look at those things or cherry pick those things, the un-cherry-picked-enough sea goes to the other half of the population of submitters to then do the same cherry pick crap the first set did. I know how the process works. It doesn't matter if it is 3 or 6. The same failure mode occurs. That failure mode, is that this process is broken because it depends on peoples personal bias. Give a full stack rank. Rank them from top to bottom. Ive explained this multiple times to Sal, and he did say he thought that would be much better. The problem is he claimed he had other things to do, and IOTD is a PAID FOR CUSTOMER FEATURE. I dont really know what is more important than what you literally list on the paid feature set for people to pay for the service. If the paid for feature does not work in the way the people that PAY FOR IT deem to be right, you know what you do? YOU FIX IT. I sell data to people all over the world, and if they have any issues with WHAT THEY PAY FOR, I FIX IT. How is this not common sense? -Bill |
7.90
#...
·
·
1
like
|
---|
Read the friggin' Terms of Service, here below posted verbatim: DESCRIPTION OF SERVICE The Service Provider may offer to provide some services and content, as described with more detail on the Site, ("Services"). Services shall include, but not be limited to, any service and content the Service Provider performs for you (including, but not limited to the display of text, user comments, messages, information, data, graphics, photographs, images, illustrations, software, also known as the "Content"). The Service Provider may change, suspend or discontinue the Services including any Content for any reason, at any time, including the availability of any feature or content. The Service Provider may also impose limits on certain features and services of restrict your access to parts or all the Services without notice of liability. Does it say anywhere in the text that there is such a thing as IOTP and you have the darn right to see your image on it at least once a year? No? End of story. |
#...
·
|
---|
Bill Long - Dark Matters Astrophotography:andrea tasselli: Bill, the underlying issue is, why do we need such a system at all? I also print my images on aluminum and have them hanging around the house. Much better than having to stare at a computer screen all the time. Also there is the aforementioned individual solution of having a personal website. Don't put so much thought in this IOTD thing, it's useless. It drives people to unnecessarily compete and argue, and for what? |
11.02
#...
·
|
---|
andrea tasselli: No one asked for that, so what you suggested here is just a strawman argument. IOTD is a paid-for feature that does not work well. The developer has confirmed this. What you posted here is just obtuse garbage that has no relevancy to the discussion at all. |
20.52
#...
·
·
4
likes
|
---|
I stopped paying much attention to IOTD a long time ago.... to be honest, I just dont care anymore and will probably regret wading into this discussion. For me, I got sick of seeing the same imagers getting IOTD over and over and over again. If you go back through the IOTD for the last year there are a handful of people who seem to get it repeatedly. Most of these are excellent images, but there are thousands of images from hundreds of outstanding imagers that are not getting it? Why are so few people getting a disproportionate high number of IOTD's? The system seems biased to me. From what I understand, the final level of promotion is not blind, so the promoters know who it is. Shouldnt an image stand on it's own? Why should a judge know who made the image at any point in the process? I know watermarks (when embedded before uploading) makes this difficult, but still.... go back through the last year and you will see that some people just seem to get IOTD over and over again... So we went from the old system where IOTD was a community popularity contest to a current system where the judges are participating in favoritism? The current biased system devalues IOTD. Scroll through the feeds and there are countless IOTD worthy images... many dont even get top pick or nominations which is really sad. Regarding the comparison to APOD. I have no problem with APOD. Thats a publication that shows cool "space" pictures to the general public. Abin is a community of imagers, and caters to imagers. IOTD should be held to a much higher standard when it comes to technical and artistic consideration. |
7.90
#...
·
|
---|
Bill Long - Dark Matters Astrophotography: If it ain't in what you willingly subscribed to then it does not exist. I rest my case. |
#...
·
·
5
likes
|
---|
Bill Long - Dark Matters Astrophotography: First the sensationalist title, now the fake news. This is completely untrue. |
11.02
#...
·
·
1
like
|
---|
AdrianC.:Bill Long - Dark Matters Astrophotography:andrea tasselli: Its not the competition I care about. I think there are images here that grossly benefit from either the imager being known, or a prejudice from the staff in terms of what they like. Mostly the last two are a fault of the judges. I vomit when I see the 6th or 7th NGC1333 or M78. That is a bias and preferece issue, and is caused by the judges being given unilateral control of where that final award goes. The voting process overall is flawed, but more importantly people could appreciate a completely blind and non-biased review of their data. It can help promote the hobby, if done right, or it can completely take a shit on it, as it is implemeted now. Make no mistake about this, the current IOTD process is laughed about, among physicists, people that work at NASA, and beyond. It is a complete joke, and it can be fixed to not be so. That is what I am advocating for. A good, real, normal voting process that does not need some over-eager person in Australia or Namibia or Washington State to decide its fate. -Bill |
12.24
#...
·
·
3
likes
|
---|
Chris White- Overcast Observatory: While there may or may not be bias, it is at least true that a large number of good images will be missed due to the very limited number of slots. There could be multiple ways to fix this problem. But as I mentioned above, you will see resistance to doing it from a relatively very small number of people for whom the current system works. A large number of other people don't even bother to participate in these discussions because they know that no meaningful change will happen. I cannot remember now the last time I looked at the list of TPs and TPNs. And I just gloss over whatever image shows up as IOTD on a particular day. |
11.02
#...
·
|
---|
Salvatore Iovene:Bill Long - Dark Matters Astrophotography: From: Salvatore Iovene <[email protected]> Sent: Saturday, April 6, 2024 6:48 AM To: Bill Long <[email protected]> Subject: Re: IOTD Problem Hi Bill, I’m aware that there’s a little bit of a Top Pick draught going on since quite some time. I’m taking measures to improve the situation as the average number of Top Pick daily is currently not deemed to be representative of the quality and quantity of images submitted for IOTD/TP consideration. Thanks for the feedback! Salvatore Oh really? Fake News? Nope. |
#...
·
|
---|
That doesn't mean that I said "it's not working well", LOL. The IOTD/TP process requires occasional fine-tuning in some parameters (number of staff members, number of voting slots, length of time for an image to be in the process, etc) as external factors change (fatigue of the teams, new team members who need to adjust, etc). |
11.02
#...
·
|
---|
Salvatore Iovene: Oh right, hmm... but here you changed the service. So clearly it was not working well. Right? From: Salvatore Iovene <[email protected]> Sent: Saturday, April 6, 2024 7:05 AM To: Bill Long <[email protected]> Subject: Re: IOTD Problem Hi Bill, once an image advanced to the next tier, Submitters will stop being able to view it, so views won’t be collected anymore. AstroBin doesn’t count Reviewer views at the moment, but it keeps record of votes of course. If a Reviewer is not active enough (used to be 7 votes per week but I just bumped it to 14) they are warned, and then removed from the team (to avoid unlucky images being in the queue of an insufficiently active team member). However, as I said, yes: there needs to be more Reviewer activity. I just added two more voting slots for them, 2 more reviewers (and I will add a few more), and made the minimum required activity 14 votes per week. Hopefully this helps! Salvatore |
11.02
#...
·
|
---|
Salvatore Iovene:Bill Long - Dark Matters Astrophotography: So this statement you made here ^^^ Is the one that is completely untrue, right? |
#...
·
·
3
likes
|
---|
Number of slots is a parameter that needs fine-tuning sometimes. If you think that a system that evolves and improves over time is broken because it's not perfect like a deity, then this is on you. |
11.02
#...
·
|
---|
Salvatore Iovene: Or you can use what every other voting system uses. Blind votes. This would be an upgrade. Stack ranking, which allows for more engagement across the data your team is asked to look at, and a way for them not to cherry pick. BLIND JUDGES: The most important part of this. Your judges know who people are. That completely undermines the entire process. All scientific evaluations of judges have shown that knowing the parties being judged, always leads to a change in their decision otherwise. Astrobin is a flawed process. You have admitted this to me (then lied to people here), changed the process, but did not fix the root cause. You should implement stack ranking. I sent you a very brief note about this. If you find this to be a challenge, I am willing to give free time to you if you want to implement this change. You should make every tier of ranking completely blind. -Bill |
11.02
#...
·
|
---|
Chris White- Overcast Observatory: People likely passed this comment over from Chris, but people should take the time to read it. This is the feedback I get from all imagers at all levels of competency. Chris owns a Maple Syrup company in Vermont. While to some that might seem odd for me to mention, stay with me. The juding process of who has the best maple syrup in Vermont is a highly contested award, after all the syrup from Vermont, is likely (if you live in the US) still in the cheapo syrup containers you buy -- its just the stuff they would discard vs sell. Nontheless, the awards are important. Not because Chris or other companies want to flex, its to help them be recognized. Astro imaging has the same game. People want to be known, and with the current state of things we see the data show that significant weight is given, by the non-blind judging panel, to those commonplace to Top Picks or IOTD. This is a fundemental flaw that needs to be fixed, as soon as possible. IOTD is a paid feature of all of the folks that sign up for that level of the service. Calling our feedback "fake news" when it isnt, or otherwise diminishing the disdain for a paid feature is poor business and should never be a part of a distinguished online service. -Bill |
4.36
#...
·
·
2
likes
|
---|
Bill Long - Dark Matters Astrophotography:Jan Erik Vallestad: Well, that's not true unless I'm missing something very obvious. I was the third person posting. Neither of the ones before were from you.. You've edited your original post now so it does comply a bit more with what I requested, that's a start I guess. |
#...
·
·
12
likes
|
---|
Bill Long - Dark Matters Astrophotography: Submitters and Reviewers already receive anonymized images. You will argue that they can go and look up the images and find the authors. The "fix" for this is to prevent images from being published while they are in the IOTD/TP, and nobody wants that. This is one of the reasons why the IOTD/TP is not a contest. On an actual contest, I would implement this. Additionally, unless you think that the whole world is against you, remember that AstroBin has other ways to combat bias, such as: - a single person only has a 50% probability to get a certain image. If you have a hater or somebody biased against you or your business, they might not get your image to vote - a single vote does nothing: you need 3. So you'd need many people who are biased against you, to make any kind of impact. I sent you more calculations of probabilities in a private message some time ago Bill Long - Dark Matters Astrophotography: This is not feasible. It would require Submitters to spend more time on images, which would cause more fatigue and make them more likely to skip things altogether. Now they need to promote the images they think deserve advancing, so it's easier to scroll past the images that aren't that great. A way to address this would be to add many more Submitters. At that point, there aren't enough volunteers that meet the minimum criteria of having read and understood and agreed with the IOTD manifesto. We'd need to completely democritize and we have a whole lot of issues to deal with. That's a major change that would require several months to design and implement, and carries a huge risk of worsening the process. Bill Long - Dark Matters Astrophotography: Judges already have to deal with bias-avoidance mechanism: they can only make IOTD one image per week. To get your image discriminated against you need that of the 30 submitters who get it in their queues, 27 are biased against you. Then you need 17 reviewers to be biased against you. Then you need all 7 judges to be biased against you. If you believe this is the case, then unfortunately there is nothing I can say that will make you change your mind. Bill Long - Dark Matters Astrophotography: What I have surely said, and I continue to stand for, is that the IOTD/TP is not a PERFECT process. I'd be mighty arrogant if I said that! But I find it hard to debate with somebody who talks in such inflamatory terms, so I have little patience for this. And you seem to have more time than me. Bill Long - Dark Matters Astrophotography: Complete blindless (one that cannot be eluded) might be an improvement. In my opinion, the current level of blindness and bias avoidance is sufficient. If people like dusty images, they will vote for those images regardless of any additional blindness in the system. If you think people are on a cruisade against you, please refer to the thing I mentioned above regarding how many people you need to be set against you for it to work. I find myself now repeating things that I told you privately once, and as much as I don't want to ignore your concerns or wave them away, I cannot continue to repeat myself. Bill Long - Dark Matters Astrophotography:Chris White- Overcast Observatory: The last 365 IOTDs were awarded to 205 distinct astrophotographers. A bit more if you include collaborations. What more do you want, really? Several people have won a few IOTDs... I wonder... is it possible that they keep putting out amazing images, as an explanation? Isn't that supporting the fact that IOTD/TP works well, by awarding great images regardless of who's submitting them? Unfortunately, it seems like you're just being too hard to please, sorry to be blunt. |