cpo_admin

joined 5 months ago
MODERATOR OF
 

Hello All,

Unfortunately we were down for 30 minutes to an hour as some of the postgres servers at our host suffered a glitch that took them offline. Things are back up and running but there may be a little bit of lag as the servers sync back up.

Apologies for the inconvenience.

[–] cpo_admin@crazypeople.online 1 points 1 month ago

I haven't had time to dig into it deeper yet and didn't want to leave things in a mess for long yesterday. Everything appeared to update successfully but when the UI was updated and restarted it had several problems that made it unusable. Anything that uses the API had no trouble reaching the site after the upgrade which leads me to believe there's an issue specific to the webui and how our instance is currently configured/hosted. I don't want to speculate too much without more information. The setup at our host isn't typical for most Lemmy instances and occasionally we run into issues that don't impact instances run from a single server or from another host.

[–] cpo_admin@crazypeople.online 4 points 1 month ago (2 children)

And we're back up on 0.19.11. That's enough instability for one day. We'll come back to this another day once we have the network issue sorted.

Sorry again for the downtime for those of you using the webui, and for the repeated restarts for our app users. As always thank you for your patience and understanding.

[–] cpo_admin@crazypeople.online 5 points 1 month ago (3 children)

Unfortunately it looks like we're going to need to roll back to 0.19.11 for now. We're seeing some network errors when trying to access the site from the webui on the 0.19.12 build that are going to take more time to investigate. Apologies, these things happen from time to time.

 

Hello all,

We’re in the process of upgrading to the latest Lemmy release. You can read about what’s changed here:

https://crazypeople.online/post/3152222

The upgrade requires taking the site offline for a few minutes, which will happen within 5 minutes of this post. The upgrade process is expected to take less than 5 minutes.

Thank you for your patience.

[–] cpo_admin@crazypeople.online 1 points 1 month ago

It took a little longer than expected to come back online, but we're back now and things look normal. Things might be a touch slow for the next couple hours as things sync up again. Long term performance should be much more stable with fewer dips in performance when new communities are added or a new instance discovers ours.

Thank you again for your patience. We're doing our best to stay ahead of the performance curve so our members can have the smoothest experience we can offer.

 

Hello all,

Now that the upgrades are complete at our host we're scaling our resources up for improved performance. The site will be unreachable for approximately 5 minutes while everything restarts so the new configuration can be applied.

Thank you for your patience.

 

cross-posted from: https://crazypeople.online/post/3361483

Terms of Service


Welcome to crazypeople.online. These Terms of Service outline the rules and guidelines for participation on our platform. By using our services, you agree to comply with these terms and contribute to a constructive community environment.

1. Community Standards

We are committed to fostering a respectful, inclusive, and safe space for all members. While open discussion and differing viewpoints are encouraged, all interactions must remain civil and free from personal attacks, harassment, or hateful conduct.

2. Prohibited Conduct

The following behaviors are strictly prohibited:

  • Harassment & Abuse: Insulting, demeaning, or targeting individuals publicly or privately is unacceptable. Allegations of harassment should be reported to moderators immediately.
  • Misinformation & Disinformation: Deliberately spreading false or misleading information is prohibited.
  • Doxing & Privacy Violations: Sharing personally identifiable information about others or encouraging self-exposure is not permitted.
  • Spam & Disruptive Behavior: Trolling, baiting, flooding, or other attention-seeking actions detract from community discourse and will not be tolerated.
  • Illegal Content: Material that violates U.S. law, local jurisdiction laws, or our NSFW policy is forbidden.

3. Content & Moderation

  • Violence & Threats: Direct calls to violence against individuals may be removed and may result in an immediate ban. Contextual evaluations apply.
  • Piracy Discussion: Discussions about piracy and links to external pirated content are permitted, provided no illegal material is hosted on our servers. Users must ensure all pirated content remains off-site.
  • Opt-Out Participation: Users may disengage from discussions at any time. Community and user-specific filters should be utilized instead of engaging in disruptive behavior.

4. Enforcement & Discretion

  • Moderators may impose additional rules within specific communities as they see fit. Our members are obliged to follow these Terms of Service in addition to any community specific rules put in place.
  • We reserve the right to remove content or ban users for violations including those not explicitly listed to preserve platform integrity. Attempting to exploit technicalities or create moderator burden may still result in disciplinary action.

5. Policy Updates

These terms may be revised as needed. Continued use of the platform constitutes acceptance of any amendments.

For questions or concerns, contact a member of the admin team. Current admins are always listed at the bottom of the sidebar from our main page.

 

Terms of Service


Welcome to crazypeople.online. These Terms of Service outline the rules and guidelines for participation on our platform. By using our services, you agree to comply with these terms and contribute to a constructive community environment.

1. Community Standards

We are committed to fostering a respectful, inclusive, and safe space for all members. While open discussion and differing viewpoints are encouraged, all interactions must remain civil and free from personal attacks, harassment, or hateful conduct.

2. Prohibited Conduct

The following behaviors are strictly prohibited:

  • Harassment & Abuse: Insulting, demeaning, or targeting individuals publicly or privately is unacceptable. Allegations of harassment should be reported to moderators immediately.
  • Misinformation & Disinformation: Deliberately spreading false or misleading information is prohibited.
  • Doxing & Privacy Violations: Sharing personally identifiable information about others or encouraging self-exposure is not permitted.
  • Spam & Disruptive Behavior: Trolling, baiting, flooding, or other attention-seeking actions detract from community discourse and will not be tolerated.
  • Illegal Content: Material that violates U.S. law, local jurisdiction laws, or our NSFW policy is forbidden.

3. Content & Moderation

  • Violence & Threats: Direct calls to violence against individuals may be removed and may result in an immediate ban. Contextual evaluations apply.
  • Piracy Discussion: Discussions about piracy and links to external pirated content are permitted, provided no illegal material is hosted on our servers. Users must ensure all pirated content remains off-site.
  • Opt-Out Participation: Users may disengage from discussions at any time. Community and user-specific filters should be utilized instead of engaging in disruptive behavior.

4. Enforcement & Discretion

  • Moderators may impose additional rules within specific communities as they see fit. Our members are obliged to follow these Terms of Service in addition to any community specific rules put in place.
  • We reserve the right to remove content or ban users for violations including those not explicitly listed to preserve platform integrity. Attempting to exploit technicalities or create moderator burden may still result in disciplinary action.

5. Policy Updates

These terms may be revised as needed. Continued use of the platform constitutes acceptance of any amendments.

For questions or concerns, contact a member of the admin team. Current admins are always listed at the bottom of the sidebar from our main page.

[–] cpo_admin@crazypeople.online 2 points 1 month ago

You're welcome! There's a lot of learning as we go and some things will always be out of our control.

Keeping everything running smoothly has been more hands on than expected with a managed hosting service. We have a good working relationship with our host but they are a relatively small operation so fixes and upgrades aren't as seamless as they probably could be. Communicating downtime windows could also use some work as we've recently seen. Still, they do a great job responding to our concerns and helping with things that require more heavy lifting than we're comfortable with.

I hope people are finding it an enjoyable place to spend their time online despite the hiccups here and there.

[–] cpo_admin@crazypeople.online 5 points 1 month ago (2 children)

The upgrades are complete and so far everything looks good. We're seeing vastly increased disk throughput and less time spent hanging while waiting for data. The instance should be feel more responsive more often now. I'm going to leave the pinned post up for now just in case anything else comes up, but I'm cautiously optimistic we are in the clear now.

 

Hello all,

As I'm sure many of you noticed we were down for over an hour. The good news is that we're back, the server upgrades that were previously delayed are being completed, and things should be more reliable going forward.

I'd like to apologize for not giving notice that this was going to happen. Unfortunately, the last update from our host indicated that the maintenance wasn't going to happen today. As far as we knew the next attempt at upgrading wasn't due to happen until the 28th. We've brought this up with our host as something that needs to be better communicated going forward. Hopefully the downtime wasn't too much of an annoyance for anyone.

As always, thank you for your patience and understanding.

[–] cpo_admin@crazypeople.online 1 points 1 month ago

This has been rescheduled yet again. Sorry for the on again off again updates.

[–] cpo_admin@crazypeople.online 2 points 1 month ago

Finally more information to pass along-

Our host has scheduled the maintenance window for 11pm-7am UTC tomorrow. It is likely the site will be completely unreachable for some portion of this time, and intermittently available for the majority of the maintenance window. Our best guess is that from 5am-6am UTC the site will be completely offline as the hardware upgrades and data migration take place.

https://time.now/timezones/utc/

[–] cpo_admin@crazypeople.online 1 points 1 month ago

The upgrade is causing some problems with our test instance and is on hold for now while we get them sorted out. Hopefully we can get it sorted and push the upgrade through here shortly.

[–] cpo_admin@crazypeople.online 2 points 1 month ago

A further update after more discussion with our host-

The major factor at play here is the increase in scrapers/bots/AI traffic to servers hosting federated content. There are large waves of bot requests that are pushing the postgres service to capacity across multiple customer sites within our host's infrastructure. Our host blocks most scrapers automatically, and is actively working to block new bots to minimize the impact of the increased traffic as much as possible. Despite those efforts we're still experiencing some problems here and there that we hope to have solved after the upgrades this weekend.

Again, thank you for understanding.

2
[Ongoing] Unplanned Downtime (crazypeople.online)
submitted 1 month ago* (last edited 1 month ago) by cpo_admin@crazypeople.online to c/cpo_announcements@crazypeople.online
 

Hello again,

We had about half an hour of unplanned downtime this morning, and there have been a few short times over the last few days when the site was down. I'd like to apologize for that as it is not the experience we intend to deliver.

For reasons that are still being investigated, the PostgreSQL servers at our host were overloaded and required manual restarting rather than the automated restart that should've happened. We're expecting some of this problem to be remedied after this weekends maintenance to upgrade to NVME drives on postgres servers. We're still looking into the problem and talking it over with our host to prevent it from becoming a regular problem now and in the future.

Thanks you for your patience and understanding as we work through these things.

[–] cpo_admin@crazypeople.online 2 points 1 month ago

There have been a few short (less than 5 minute) service interruptions over the last couple days related to the coming upgrade at our host. We often aren't given any warning if the downtime is expected to be very brief. Apologies if the experience is less than stellar. That isn't what we want to provide.

Hopefully things will be running smoother than ever after this weekends maintenance.

[–] cpo_admin@crazypeople.online 2 points 1 month ago

The adjustments have been completed and everything looks good. The next expected downtime will be for our storage upgrades on June 14th.

 

Hello again all,

We're making a few more adjustments to our Pict-rs configuration that require a quick restart. These usually take under a minute from start to finish. During this time the site will be unreachable for local users.

Click here for current UTC Time

Thank you for your patience.

 

We are making adjustments to our Pict-rs image proxy that will require a quick server restart. These usually take under a minute and aren't expected to cause disruption for more than a few seconds. Sorry for the short notice and thank you for understanding.

2
NSFW Content Policy (crazypeople.online)
submitted 1 month ago* (last edited 1 month ago) by cpo_admin@crazypeople.online to c/main@crazypeople.online
 

NSFW Content Policy


Purpose and Scope:

This policy governs the handling of Not Safe For Work (NSFW) content across our platform, establishing clear guidelines for both locally hosted and remote content. Our primary objective is to maintain a balanced environment while ensuring responsible content management.

Content Classifications


NSFW Categories:
  1. Locally Prohibited Content
  • Pornographic material in any form
  • Sexually explicit content
  • Content that violates applicable laws or regulations
  1. Locally Allowed Content
  • Adult-themed content not classified as pornographic
  • Content requiring appropriate content warnings
  • Material that may be inappropriate for general viewing

Content Jurisdiction


Local Content

Local content encompasses all material directly hosted on our servers, including:

  • Posts and uploads to our Lemmy instance
  • Images and media stored on our Pict-rs servers
  • User-generated content hosted within our infrastructure

All local content must strictly adhere to this policy's guidelines. The admin team maintains absolute discretion in determining content appropriateness and reserves the right to remove any material deemed unsuitable.

Remote Content

Remote content refers to material hosted on external instances or third-party platforms. Our policy regarding remote content is as follows:

  • Users may subscribe to remote NSFW communities
  • Direct posting or uploading of NSFW content to remote locations via local accounts is prohibited when such actions would result in media storage on our servers
  • The admin team reserves the right to restrict access to remote content that violates our guidelines

Enforcement Protocol


Violation Response Framework

Our instance employs a graduated response system for policy violations:

  1. Content Removal
  • Immediate removal of violating material
  • Policy reference provided to the user
  • Documentation of the incident
  1. Temporary Suspension
  • Implemented for repeated violations
  • Applied after previous warnings
  • Duration determined by violation severity
  1. Account Termination
  • Reserved for severe or repeated violations
  • Automatic implementation for illegal content
  • Permanent restriction from platform access

Administrative Rights


The administration team reserves the right to:

  • Modify this policy without prior notice
  • Remove any content at their discretion
  • Implement disciplinary actions as deemed necessary
  • Make final determinations on policy interpretation

Implementation

This policy is effective immediately and applies to all existing and future content on the platform. Users are responsible for reviewing and complying with these guidelines.

For questions or clarification regarding this policy, please contact the administration team.

1
NSFW Content Policy (crazypeople.online)
submitted 1 month ago* (last edited 1 month ago) by cpo_admin@crazypeople.online to c/cpo_announcements@crazypeople.online
 

NSFW Content Policy


Purpose and Scope:

This policy governs the handling of Not Safe For Work (NSFW) content across our platform, establishing clear guidelines for both locally hosted and remote content. Our primary objective is to maintain a balanced environment while ensuring responsible content management.

Content Classifications


NSFW Categories:
  1. Locally Prohibited Content
  • Pornographic material in any form
  • Sexually explicit content
  • Content that violates applicable laws or regulations
  1. Locally Allowed Content
  • Adult-themed content not classified as pornographic
  • Content requiring appropriate content warnings
  • Material that may be inappropriate for general viewing

Content Jurisdiction


Local Content

Local content encompasses all material directly hosted on our servers, including:

  • Posts and uploads to our Lemmy instance
  • Images and media stored on our Pict-rs servers
  • User-generated content hosted within our infrastructure

All local content must strictly adhere to this policy's guidelines. The admin team maintains absolute discretion in determining content appropriateness and reserves the right to remove any material deemed unsuitable.

Remote Content

Remote content refers to material hosted on external instances or third-party platforms. Our policy regarding remote content is as follows:

  • Users may subscribe to remote NSFW communities
  • Direct posting or uploading of NSFW content to remote locations via local accounts is prohibited when such actions would result in media storage on our servers
  • The admin team reserves the right to restrict access to remote content that violates our guidelines

Enforcement Protocol


Violation Response Framework

Our instance employs a graduated response system for policy violations:

  1. Content Removal
  • Immediate removal of violating material
  • Policy reference provided to the user
  • Documentation of the incident
  1. Temporary Suspension
  • Implemented for repeated violations
  • Applied after previous warnings
  • Duration determined by violation severity
  1. Account Termination
  • Reserved for severe or repeated violations
  • Automatic implementation for illegal content
  • Permanent restriction from platform access

Administrative Rights


The administration team reserves the right to:

  • Modify this policy without prior notice
  • Remove any content at their discretion
  • Implement disciplinary actions as deemed necessary
  • Make final determinations on policy interpretation

Implementation

This policy is effective immediately and applies to all existing and future content on the platform. Users are responsible for reviewing and complying with these guidelines.

For questions or clarification regarding this policy, please contact the administration team.

 

Hello again all, hope you're well.

Saturday June 14th crazypeople.online will be going down for maintenance work taking place at our host. During this time the site will be unreachable for local users. The exact shutdown time and expected time to complete the upgrade are still be TBD. This post will be updated when we have more specific information to pass along.

There are several upgrades taking place at this time, most importantly the switch to NVME storage for our PostgreSQL servers. This is expected to drastically increase our I/O throughput, which should provide better performance across all the services we offer as they all interact with PostgreSQL continuously.

Keep an eye on this community for further updates.

view more: next ›